


Institutional Archive of the Naval Postgraduate School 





Calhoun: The NPS Institutional Archive 


DSpace Repository 


Theses and Dissertations 


2009-09 


1. Thesis and Dissertation Collection, all items 


Collaborative online communities for 
increased MILSATCOM performance 


Holgerson, Jason L. 


Monterey, California. Naval Postg 


http://ndl.handle.net/10945/4523 


Downloaded from NPS Archive: C 


atthe | DUDLEY 


WN] | ciseaRy 


http://www.nps.edu/library 





raduate School 


alhoun 


Calhoun is the Naval Postgraduate School's public access digital repository for 

research materials and institutional publications created by the NPS community. 

Calhoun is named for Professor of Mathematics Guy K. Calhoun, NPS's first 
appointed — and published — scholarly author. 


Dudley Knox Library / Naval Postgraduate School 
411 Dyer Road / 1 University Circle 
Monterey, California USA 93943 





NAVAL 
POSTGRADUATE 
SCHOOL 


MONTEREY, CALIFORNIA 


COLLABORATIVE ONLINE COMMUNITIES FOR 
INCREASED MILSATCOM PERFORMANCE 
by 


Jason L. Holgerson 


September 2009 


Thesis Advisor: John Osmundson 
Second Reader: Mark M. Rhoades 





Approved for public release; distribution is unlimited 


THIS PAGE INTENTIONALLY LEFT BLANK 


REPORT DOCUMENTATION PAGE 


Public reporting burden for this collection of information is estimated to average | hour per response, including the time for reviewing instruction, 
searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send 


comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to 
Washington headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 
22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188) Washington DC 20503. 


1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED 
September 2009 Master’s Thesis 

4. TITLE AND SUBTITLE 5. FUNDING NUMBERS 

Collaborative Online Communities for Increased MILSATCOM Performance 


6. AUTHOR(S) Jason L. Holgerson 


7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION 
Naval Postgraduate School REPORT NUMBER 
Monterey, CA 93943-5000 


9. SPONSORING /MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITORING 
N/A AGENCY REPORT NUMBER 








11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the official policy 
or position of the Department of Defense or the U.S. Government. 


12a. DISTRIBUTION / AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE 
Approved for public release; distribution is unlimited A 


13. ABSTRACT (maximum 200 words) 








The Department of Defense and, subsequently, the U.S. Navy have embraced a strategy of exerting influence through 

information dominance versus amassing a large presence. This philosophy, called Net-centric Warfare, uses sensor 
and network technology to leverage naval platforms towards realizing effects previously achievable only by a larger 
force. 

In adapting this strategy, the U.S. Navy has realized many benefits, but has also increased its reliance on the 
technologies implementing Net-centric Warfare. One such technology is the U.S. Navy’s future Military SATCOM 
terminal, the Navy Multiband Terminal, which will provide critical off-ship bandwidth required for these leveraging 
effects. The Navy plans to sustain the NMT system using the same methods as previous systems. 

When considering Operational Availability, these legacy methods do not adequately address attributes 
affecting system performance, creating a risk the NMT system will not perform as needed to successfully execute 
Net-centric Warfare. This risk can be managed by transitioning away from traditional methods to those utilizing 
online collaborative technologies. These technologies, coined “Web 2.0,” center around member participation to 
foster communities. Much like the philosophy of Net-centric warfare, these communities leverage the experience of 
individuals to the benefit of the entire community. 





14. SUBJECT TERMS 15. NUMBER OF 
Operational Availability, Net-centric Warfare, Communities, Online, Web 2.0, Sustainment, Military PAGES 
SATCOM, NMT 99 


16. PRICE CODE 


17. SECURITY 18. SECURITY 19. SECURITY 20. LIMITATION OF 
CLASSIFICATION OF CLASSIFICATION OF THIS CLASSIFICATION OF | ABSTRACT 





REPORT PAGE ABSTRACT 
Unclassified Unclassified Unclassified UU 





NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) 
Prescribed by ANSI Std. 239-18 


THIS PAGE INTENTIONALLY LEFT BLANK 


ll 


Approved for public release; distribution is unlimited 


COLLABORATIVE ONLINE COMMUNITIES FOR INCREASED 
MILSATCOM PERFORMANCE 


Jason L. Holgerson 
Assistant Program Manager 
PEO C4I 
San Diego, California 
B.S., University of Colorado, 1999 


Submitted in partial fulfillment of the 
requirements for the degree of 


MASTER OF SCIENCE IN SYSTEM ENGINEERING MANAGEMENT 


from the 


NAVAL POSTGRADUATE SCHOOL 


September 2009 
Author: Jason L. Holgerson 
Approved by: John Osmundson 
Thesis Advisor 


Mark M. Rhoades 
Second Reader 


David H. Olwell 
Chairman, Department of Systems Engineering 


i 


THIS PAGE INTENTIONALLY LEFT BLANK 


1V 


ABSTRACT 


The Department of Defense and, subsequently, the U.S. Navy have embraced a 
strategy of exerting influence through information dominance versus amassing a large 
presence. This philosophy, called Net-centric Warfare, uses sensor and network 
technology to leverage naval platforms towards realizing effects previously achievable 
only by a larger force. 

In adapting this strategy, the U.S. Navy has realized many benefits, but has also 
increased its reliance on the technologies implementing Net-centric Warfare. One such 
technology is the U.S. Navy’s future Military SATCOM terminal, the Navy Multiband 
Terminal, which will provide critical off-ship bandwidth required for these leveraging 
effects. The Navy plans to sustain the NMT system using the same methods as previous 
systems. 

When considering Operational Availability, these legacy methods do not 
adequately address attributes affecting system performance, creating a risk the NMT 
system will not perform as needed to successfully execute Net-centric Warfare. This risk 
can be managed by transitioning away from traditional methods to those utilizing online 
collaborative technologies. These technologies, coined “Web 2.0,” center around 
member participation to foster communities. Much like the philosophy of Net-centric 
warfare, these communities leverage the experience of individuals to the benefit of the 


entire community. 
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EXECUTIVE SUMMARY 


The Department of Defense and, subsequently, the U.S. Navy have embraced a 
philosophy, called Net-centric Warfare, using sensor and network technology to leverage 
individual naval platforms to realize effects previously achievable only though a larger 


force. 


Adapting this strategy, the U.S. Navy has realized many benefits but has 
increased the reliance on technologies implementing Net-centric Warfare. One such 
technology is the U.S. Navy’s future Military SATCOM terminal, the Navy Multiband 
Terminal, which replaces two systems currently providing critical off-ship bandwidth 
essential for implementing the Net-Centric doctrine. However, despite the increased 
criticality, the NMT system threshold requirement for availability—a key performance 
parameter for bandwidth—is 90%, commensurate with the two systems NMT is 
replacing. This threshold requirement is not consistent with the expectation for 
availability, which is closer to the NMT objective requirement of 99%. To overcome the 
spread between threshold and objective requirements, the NMT system is scheduled to 
implement the same data collection methods as these prior systems. This thesis defines 
this inherited system and evaluates the suitability of this system towards improving 
system availability. Further, the thesis examines the emergence of online collaborative 
communities, proposes a new system using these technologies, and evaluates this 


proposed system’s suitability towards improving NMT availability. 


The current method for performance data collection is a hierarchical system with 
a focus on reporting current material status rather than long-term system availability 
improvement. An anomaly experienced by a System Operator initiates a process of 
diagnosis and corrective action that extends from them to the ISEA Technicians through 
the System Maintainers and RMC Technicians. Along this chain, the System Maintainers 
and RMC Technicians predominantly utilize the 4790/2K to record information. 
However, this form 1s heavily focused on reporting current material readiness to the chain 


of command, critical information but of limited value to the long-term improvement of 


xi 


the system. In addition to the 4790/2K, problems are reported via a Casualty Report, but 


again, this is focused more on readiness reporting than system improvement. 


An ideal data collection system focused on improving system availability is 
postulated to serve as a basis of evaluation for the current system and an intermediary for 
comparing the current system to a proposed system. The ideal system is comprised of 
nine attributes, all of which if addressed would improve system availability. When 
evaluated against the ideal system’s nine attributes, the current system is found to 
perform almost as well as the ideal in one attribute, marginally address five attributes, 
and effectively fail to address three attributes. Driving this score is the fact that the 
current system limits who can document information, what information can be 


documented and, once documented, with whom the information is shared. 


These constraints run counter to the philosophy of “Web 2.0,” which is the 
application of online Web technologies for the development and sharing of information 
amongst users. The broad participation in development and consumption of information 
has facilitated formation of communities that defy traditional geographic constraints. 
Deployed in various models, these technologies have resulted in well-known services, 
such as YouTube, Wikipedia, and Facebook, that support communities consisting of 


millions of users exchanging billions of pieces of information. 


While the NMT user community may not number into the millions, an 
opportunity exists to employ these technologies to establish an NMT user community. 
Considering the roles of the community members, a model that combines multiple Web 
2.0 technologies is proposed for the NMT community. This model provides a 
mechanism for members with the most frequent NMT interaction, the System Users and 
System Maintainer, to record their experiences and problems with the NMT system in a 
blog. Other members are able to assist the System Users and System Maintainers by 
providing comments to these blog postings. The blog postings and comments are 
archived, searchable and shared to all members of the community, providing an 


opportunity for peer-to-peer training. In addition to increasing the competency of the 


X1V 


users, the creation and exchange of information provides the Acquisition Program Office 
a broad base of information upon which to make follow-on investments affecting system 


availability. 


In contrast to the current system that creates silos of information by restricting 
access, the proposed system based upon Web 2.0 technologies encourages and thrives on 
the broad creation and sharing of information. This difference becomes evident when the 
proposed system is evaluated against the ideal, which allows for a comparison between 
the proposed and current systems. The proposed system was found to perform almost as 
well as the ideal system in all nine attributes, a stark difference to the current system, 


which achieved this score for only one attribute. 


The relative poor performance of the current system does not guarantee a failure 
to adequately support the NMT system, but given the potential of the proposed system, 
continued us of the current system does seem to be an imprudent risk. With this in mind, 
it is clear the NMT program should seek to exploit the opportunities created by Web 2.0 
technologies to ensure the NMT system provides critical bandwidth commensurate with 


the expectations of its user and Net-centric Warfare. 
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I. INTRODUCTION 


A. BACKGROUND 


“Future naval operations will use revolutionary information superiority and 
dispersed, networked, force capabilities to deliver unprecedented offensive power, 
decisive assurance, and operational independence to Joint Force Commanders” (Clark, 
2002). This idea, often called Network-centric warfare, has represented a fundamental 
change in the way the Department of Defense (DoD) plans and conducts military 
operations. Under this idea, “the fundamental approach of warfare has changed from 
massing forces to massing effects” (Joint Chiefs of Staff, 1995). Through the broad 
collection, dissemination, synthesis, and analysis of information, the Navy is able to 
leverage the impact of an individual platform to exert an influence only previously 
achievable through a large naval presence. Employing this leveraging philosophy has 
provided the Navy with several benefits, the most notable being an increased lethality via 
a smaller, more affordable fleet. However, while providing tremendous benefit, the Navy 
is now critically reliant on the systems that enable this capability. One such system 1s the 


Navy Multiband Terminal (NMT). 


NMT is the Navy’s next generation, and only future Military Satellite 
Communications (MILSATCOM) terminal. The NMT consolidates the same SATCOM 
capability provided by two previous systems into a single system. In a single terminal, the 
NMT will provide the protected Extremely High Frequency (EHF) capabilities of the 
AN/USC-38(V), the wideband Super High Frequency (SHF) capabilities of the 
AN/WSC-6(V), and the receive capabilities of the Global Broadcast System (GBS) 
antenna system. As depicted in Figure 1, NMT’s Department of Defense Architecture 
Framework (DODAF) Operational View | (OV-1), the NMT will provide ship, shore, 
and submarine communication capabilities to most of the DoD’s current and future 


military satellite constellations. 


Figure 1: NMT DODAF OV-1 





NMT Users 


Given the obvious absence of terrestrial connectivity in the maritime 
environment, the Navy has a tremendous reliance on Satellite Communications 
(SATCOM); this reliance 1s compounded by the Network-centric warfare philosophy. 
However, the Availability (Ao) requirement, probability the system will able to support 
tasking, for the NMT system has not increased commensurately with its importance. 
Table 1 enumerates the Ao requirements for the NMT system along with the AN/USC- 
38(V) and AN/WSC-6(V) systems, which the NMT system will replace. As depicted by 
the table, despite the increased criticality of the NMT system, the threshold requirements 
are essentially unchanged. However, Table | also shows an objective Ao of 0.99, 
creating an opportunity to achieve the availability levels commensurate with the reliance 


on NMT and shipboard expectations. 


Table 1: MILSATCOM Availability Requirements 


Threshold Ao Objective Ao 


To exploit this opportunity, and provide the Navy with high availability 





MILSATCOM, several factors must be taken into consideration. Specifically, the NMT 
must be highly reliable, easy to repair, and supported by an efficient supply chain. While 
these factors are heavily influenced by the initial design of the NMT and associated 
support systems, they are also influenced by the efficiency at which deficiencies in these 
initial designs are identified and corrected. In turn, deficiency identification and 
correction is reliant upon the quality and availability of data describing operational 
performance. For instance, clearly an unknown problem, one experienced but not 
reported, cannot be rectified. Also, the pervasiveness of the problem is also highly 
relevant to getting the biggest ‘bang for the buck’ when making resource decisions; 
frequently, as the saying goes, a mountain can be made out of a mole hill, but the inverse 
is also true. Simply, quality, comprehensive, operational data will be required for the 
NMT system to achieve Ao performance close to objective numbers and Fleet 
expectations. 

Currently, the NMT program will receive this data through the channels and 
methods of its predecessors, the AN/USC-38(V) and AN/WSC-6(V) programs. This 
thesis examines the efficacy of this plan and explores the possibilities provided by recent 
developments in Web-based collaborative technologies and approaches to develop a 


better solution to exploit the opportunity for high NMT availability. 


B. RESEARCH OBJECTIVES 


The objective of this research is to define and analyze the operational 
performance data collection methods used on today’s MILSATCOM systems, the 
AN/USC-38(V) and AN/WSC-6(V) programs. Further, with this understanding and 
analysis, evaluate the efficacy of these methods to support the NMT program in light of 
the increased necessity of availability brought on by both the broad DoD’s strategic shift 
and the Navy’s consolidation strategy. Lastly, the research will survey the recent 
developments in online, or Web-based, collaborative technologies to determine if, with 
these technologies, there is an opportunity to improve, augment, or replace the methods 


currently used. 


C. RESEARCH QUESTIONS 


1. Primary Research Question 


What are the current methods for collecting operational performance data on 
the AN/USC-38(V) and AN/WSC-6(V) MILSATCOM systems, and how well do 
they satisfy their objectives? 


Zi Subsidiary Research Questions 


What are the available online, Web-based, collaborative technologies? 


Is there an opportunity for these collaborative technologies to improve, augment, 


or replace the current methods for collecting operational data to yield better results? 


What would a proposed method for collecting NMT operational performance data 
look like given an understanding of both legacy methods and the newer Web-based 


technologies? 


D. SCOPE 


The scope of this thesis describes the current methods used to support the 


AN/USC-38(V) and AN/WSC-6(V) MILSATCOM systems, along with an evaluation of 


4 


their performance. This description will include the official methods, those dictated by 
policy, formal guidance, or generally accepted as the standard procedure. Ad hoc 
methods are not considered part of the current official support and data collection system 
for evaluation purpose. This evaluation exclusion is because ad hoc methods are neither 
pervasive throughout the system nor is there any accountability for their success or 
failure. However, these solutions will be considered when considering a proposed 
method for collecting NMT operational performance data. This proposed method will be 
built upon the survey of Web-based collaborative technologies presented in this thesis 


and the understanding of current methods. 


E. METHODOLOGY 


The methodology for this thesis consists of the following steps: 


e Conduct a policy and formal guidance review along with interviews to 
document the method used to support the AN/USC-38(V) and AN/WSC-6(V) 
MLSATCOM systems. 


e Postulate attributes of an ideal data collection system and evaluate the current 


methods against this ideal. 


e Survey the landscape of Web-based collaboration technologies through a 
review of published and online literature, best practices documents, and case 
studies. From this survey, identify technologies or applications that may be 


beneficial in the support of the NMT system. 


e Developed a proposed method of collecting NMT performance data and 


perform a forward-looking evaluation against the ideal. 


F. ORGANIZATION 


This thesis consists of six chapters. 


e Chapter I serves as an introduction to the scope and objectives of the thesis 


e Chapter II depicts the current methods that support and report the performance 
of the AN/USC-38(V) and AN/WSC-6(V) MILSATCOM systems; the 
methods the NMT program is currently planned to inherit. 


e Chapter III postulates attributes of an ideal system for support of 
MILSATCOM systems and evaluates the methods identified in Chapter II 


against this ideal. 


e Chapter IV provides a survey of the current online, Web-based, technologies 


available. 


e Chapter V proposes a new method to support the gathering of performance 
data for the NMT system based upon identified deficiencies in the baseline 
approach and opportunities discovered in the survey of available online, Web- 


based technologies. 


e Chapter VI presents conclusions and recommendations. 
G. BENEFITS OF RESEARCH 


This thesis proposes a different method for the collection of performance data 
during the deployment and operation of the NMT system. This different method will 
allow the NMT system to use applications and technologies in broad use rather than 
simply inherited methods. This new method will decrease the time taken to discover 
deficiencies in the NMT itself and its support systems. Further, this new method will 
facilitate more efficient allocation of NMT program resources to initiate corrective 
actions. Both of these, rapid feedback and efficient allocation of resources, will increase 
the availability of the NMT system facilitating realization of the DoD’s Network Centric 


Warfare vision. 


I. CURRENT METHODS FOR REPORTING MILSATCOM 
PERFORMANCE 


A. BACKGROUND 


The NMT system will replace the existing AN/USC-38(V) and AN/WSC-6(V) 
systems onboard U.S. Naval platforms to provide an array of MILSATCOM capability 
from high bandwidth wideband communications to protected communications. While the 
NMT consolidates these two missions into a single terminal, the functions, architectures, 


and physical components of these three systems are very similar. 


Driven by the technical similarities, the NMT system is planned to inherit a 
similar construct for system training and operation. NMT, like the AN/USC-38(V) and 
AN/WSC-6(V), will train sailors utilizing a curriculum incorporated into the Navy’s 
schoolhouses, called the Fleet Training Centers (PMW 170, 2008). Upon entering 
training, the new system user will assume one of two roles based upon their job 
classification and prior training. Trained to be operators of the system will be sailors 
with prior experience in networks and communications; these sailors have the 
Information Technology (IT) designation (PMW 170, 2008). The System Operators will 
be responsible for initiating, monitoring, and terminating the communication services 
enabled by the NMT system. Trained to be maintainers of the system will be sailors with 
prior experience in electronics theory and repair; these sailors have Electronics 
Technician (ET) designation (PMW 170, 2008). The System Maintainers are responsible 
for performing preventative maintenance in accordance with prescribed schedules, and, in 
the advent of a failure corrective maintenance. The number of sailors assigned to these 
roles varies between various classes of ships!, but a deployed system must have at least 


one operator and maintainer assigned, and with the exception of the submarine platforms, 


! For instance, an aircraft carrier (CVN) platform, have larger divisions dedicated the communication 
operations than a smaller platform such as a destroyer (DGG) platform. 
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this is always two different individuals?. Utilizing the NMT to provide communication 
services to the operational Navy platform will require close coordination between these 


individuals. 


In addition to training and operation, the methods for performing corrective 
maintenance are also planned to be inherited from the AN/USC-38(V) and AN/WSC- 
6(V) systems. In these corrective maintenance procedures are the data gathering 


opportunities to understand and improve the NMT’s performance and availability. 
B. RESOLUTION OF A FAILURE ONBOARD A U.S. NAVY PLATFORM 


The methods for resolving problems with the NMT system, while inherited from 
the AN/USC-38(V) and AN/WSC-6(V) systems, are not unique to MILSATCOM 
systems. These procedures are predominantly defined by two policy documents, the 
NAVSEA 4790.8B and the COMFLTFORCOMINST 4790.3B. The NAVSEA 4790.8B 
policy, titled ‘Ships’ Maintenance and Material Management (3-M) Manual’, defines the 
maintenance management procedures for the Navy. The COMFLTFORCOMINST 
4790.3B, titled ‘Joint Fleet Maintenance Manual’ defines the maintenance policies and 
responsibilities for the Navy. These two documents, augmented by procedures of various 
stakeholders, define the process for resolving problems and collecting the associated 


performance information about the NMT system. Figure 2 depicts this process. 


2 For submarines, due to obvious space constraints, these roles have been consolidated into a single 
individual. 
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Figure 2: MILSATCOM Failure Resolution Process 
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1. Shipboard Actions 


The NMT system is installed onboard Navy platforms as a mechanism for the 
platform to fulfill its mission requirements. The Commanding Office (CO), and 
subsequently his/her crew is responsible for the readiness and performance of that 
platform to fulfill its assigned mission (NAVSEA, 2003). As a result, the ship, the 
System Operator, and the System Maintainer are critical stakeholders in the problem 


resolution and documentation process. 


As Figure 2 depicts, the process is initiated when the System Operator 
experiences a failure. The guidance documentation does not explicitly define the term 
failure, but broadly, this can be considered as an inability to utilize the system in its 
intended manner to accomplish the mission. Failure cause can range from deficient 
system components to operator error, both will preclude the NMT from performing as 


intended. Once the operator experiences a failure, keeping in mind the dual system roles 


of operator and maintainer, the System Maintainer is notified to initiate corrective action. 
The steps taken by System Maintainer and the data recorded is dependent upon two 
variables, the time taken to correct the deficiency and whether the corrective action 


required a change in system configuration. 


Short duration corrective actions completed without a change in system 
configuration do not require the recording of failure data. For example, a failure 
condition may exist where the NMT is unable to provide communication services 
because one of its software data files has become corrupted. With archived versions of 
these files stored onboard, the System Maintainer could simply reload and replace the 
corrupted files. Assuming reloading the file corrects the deficiency, neither the System 


Maintainer nor the System Operator records information about the failure incident. 


Corrective actions that cannot be completed rapidly, but do not result in a 
configuration change, require the completion of a 4790/2K. Per the Ships’ Maintenance 
and Material Management (3-M) Manual, “The NAVSEA 4790/2K Form is used for 
reporting deferred maintenance actions, and the completion of those maintenance actions 
that do not result in a configuration change” (NAVSEA, 2003). For instance, if the ship’s 
NMT system experienced a hardware failure, but the ship did not have a replacement part 
in their spares inventory, the 4790/2K would record the outstanding corrective action; 
once the part is received, resulting in an operational system, the 4790/2K would be closed 
(NAVSEA, 2003). It is clear, the 4790/2K serves two purposes, one to document the 
failure event and the other to provide the ship’s chain of command a status of the ship’s 


current capabilities and deficiencies (NAVSEA, 2003). 


The 4790/2K fulfills these two purposes as part of the ship’s Maintenance Data 
System (MDS), which is the ship’s central repository for maintenance information. The 
ship’s Commanding Officer (CO) is accountable for ensuring the CMSP is current and 
complete. For entry into the MDS, the System Maintainer completes the 4790/2K form 
shown in Figure 3. Typically, this is an automated process utilizing a software 
application, but there are sites where the software application is not available (NAVSEA, 


2003). 
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Figure 3: OPNAV 4790/2K 
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The entries in the MDS are synchronized with a central shore data repository 
managed by the Navy Sea Systems Command Logistics Center (NAVSEALOGCEN), 


providing various levels of Navy leadership visibility into the current maintenance 


1] 


disposition of the fleet. As can be seen in Figure 3, the ship maintainer inputs 
information relevant to the failure, but the focus is on readiness reporting. However, if 
the System Operator desires to collect and document more information about the failure 
itself, a 4790/2L form can be completed. Per the Ships’ Maintenance and Material 
Management (3-M) Manual, “The NAVSEA 4790/2L is used to provide amplifying 
information (such as drawings and listings) related to a maintenance action, reported on a 
NAVSEA 4790/2K Form. The 2L may be used to list multiple item serial numbers and 
locations for which identical maintenance requirements exist from an outside activity; or 
to provide a list of drawings and sketches that would be helpful in the accomplishment of 
the maintenance” (NAVSEA, 2003). Once completed, the 4790/2L form is retained by 
the ship; it is not loaded into the MDS system nor is it synchronized to the shore data 
repository (NAVSEA 2003). In summary, a deferred maintenance action, which is 
ultimately resolved without a change in system configuration, results in two data 
opportunities, the 4790/2K, which is distributed to a central repository and a 4790/2L, 
which is retained by the ship. 


The deferred maintenance actions that do result in a change of the ship’s NMT 
configuration produce an addition data opportunity. These instances require the 4790/2K 
and if desired, the 4790/2L entries, but also require the completion of a 4790/CK. “The 
NAVSEA 4790/CK Form is used to report completion (or partial completion) of 
alterations, maintenance actions that resulted in a configuration change, and to correct 
discrepancies and errors in the configuration files’ (NAVSEA, 2008). Whereas the 
4790/2K is designed to provide the chain of command with visibilities into the current 
materiel status of the fleet, the 4790/CK is designed to provide the chain of command 
with visibility into the materiel capability of the fleet. For instance, different weapon 
systems provide different capabilities; when planning a mission, the planners must know 
what fleet assets have which capabilities in developing plans and assigning tasking. The 
mission planners will use the MDS populated by the 4790/CKs to determine the ship’s 
capability and then query the MDS populated by 4790/2Ks to ensure the capability is 
operational. In this capacity, the 4790/CK covers a broad array of actions from major, 


planned ship alterations to minor unplanned maintenance actions (NAVSEA, 2003). A 
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4790/CK will be completed when a ship is outfitted with the NMT system, and another 
be completed when the configuration of the NMT system is changed. If, for instance, the 
ship experiences a failure and the corrective action requires replacement of the failed 
system component with a newer, different version, a 4790/CK would be completed. 
Since a newer system component is available, presumably updated to address the 
experienced failure mode, the 4790/CK provides very little novel information about the 


system performance. 


The last scenario for the System Maintainer is corrective events that are short in 
duration, but result in a configuration change. The actions require the completion of 
4790/CK, but do not require the completion of either a 4790/2K or a 4790/2L. These 
events typically occur via one of two means; the foundation of both is an understanding 
of the failure mechanism with updated system components available. In one instance the 
ship may receive the updated system component as an augment to their spares inventory 
with the direction to install the component in the event that the existing, older version, 
component fails. This scenario, typically employed when failure modes don’t result in 
collateral damage and the criticality is low, allows the Navy to reap the most utility out of 
materiel procurements while mitigating an availability risk. The other instance is when a 
component is replaced in a proactive approach. These are scenarios where criticality is 
high, the collateral damage is high, or both. This is seen frequently in the aviation 
community, where an inventory of aircraft will be grounded until an aircraft is upgraded. 
For MILSATCOM equipment, these proactive, aggressive replacement of system 
components occur, but, unless sailor safety is involved, without the “grounding” the 
entire system inventory. With the completion of only a 4790/CK, these proactive actions 


provide very little novel information about the system performance. 


Table 2 summarizes the data opportunities available based upon the 
characteristics of the maintenance event. All of the data from these opportunities is 
provided by the System Maintainer, but there are instances where the System Maintainer 


is unable to diagnose the failure’s cause, and subsequently, unable to perform the 
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corrective action. In these instances the System Maintainer completes a 4790/2K to 
document the outstanding deficiency while waiting for outside assistance from the 


Regional Maintenance Center (RMC). 


Table 2: Data Opportunities for Shipboard Actions 


| Differed No | NoChange a Ed 


—— a 
—_ 


pa Regional Maintenance Center Actions 





The Regional Maintenance Centers (RMCs) are, as the name implies, Navy 
organizations that provide corrective maintenance assistance to operational Navy 
platforms with responsibilities for a specific geographic area; these geographic areas are 
generally coincident with Navy ports or major operational areas. Operational platforms 
are assigned to the RMC located in their homeport, but when on deployment and in 
another geographic area, can request support from the local RMC, who will, in turn 
communicate with the platform’s assigned RMC. Per Navy policy, the RMCs are the 
primary and first point of contact for any corrective maintenance assistance 
(COMFLTFORCOM, 2009). As a Navy organization, the RMCs are staffed by active 
duty military personnel, government civilians, and support contractors. Typically, 
personnel providing assistance to the ship are senior enlisted, in the case of the active 
military staff, and prior senior enlisted for the government civilian and contractor 


personnel. Support from an RMC is solicited by issuing a Fleet Technical Assist (FTA). 
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An FTA can be requested by telephone, e-mail, or official Navy message, all of 
which initiate a two stage corrective action process. Heavily motivated by cost 
considerations, distance support is encouraged and required (COMFLTFORCOM, 2009). 
In this stage of the process, the RMC representatives coordinate with the System 
Maintainer via e-mail, telephone, chat, or other communications means to understand the 
problem and assist in the failure diagnosis process. This can be a highly iterative process 
of measure, report, and assess, common with all troubleshooting activities, but 
compounded in difficulty by communication media and, in the case of large geographic 
separation, significant time differences. If this first stage of assistance is unsuccessful, 
the second stage, a visit to the operational site is initiated. The triggers to initiate the 
second stage are varied and without specific definition, rather the attributes of the 
situation and the opinion of the RMC technician to determine the necessity and 


appropriate time to initiate stage two (COMFLTFORCOM, 2009). 


Regardless of whether the FTA is resolved via distance support or an onsite visit, 
support from the RMC results in single recorded data opportunity. Maintenance actions 
performed by the RMC will be recorded in a 4790/2K (COMFLTFORCOM, 2009). Upon 
receipt of an FTA, a 4790/2K will be opened for the duration of the technical assistance. 
Like the ship initiated 4790/2Ks, the RMC initiated 4790/2K will populate the MDS to 
reflect the current materiel status of the system and operational site. Once the corrective 
action has been successfully completed, and the system returned to an operational status, 
the 4790/2K will be updated to reflect a completed service action. However, if the RMC 
is unable to resolve the issue, and return the system to operational condition, they too can 


solicit additional help from the system’s In Service Engineering Agent (ISEA). 


5 In-service Engineering Agent Actions 


United State Code (USC) Title 10 empowers a Program Manager to ensure that an 
acquisition program meets its cost, schedule and performance requirements throughout 
the lifecycle. For MILSATCOM systems in the sustainment phase of their lifecycle, a 
Program Manager tasks an ISEA to assist in ensuring Title 10 responsibilities are 


fulfilled. The ISEA supports the Program Manager through an array of tasking including 
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inventory management, updating technical publications, follow-on system development, 
and providing technical assistance. In addition, the ISEA typically has a contractual and 
close working relationship with the system’s developer, or Original Equipment 
Manufacturer (OEM). With a staff of engineers and senior technicians and close ties to 


the OEM, the ISEA is the system expert. 


To assist an operational platform in failure diagnosis and corrective action ISEA, 
support must be requested from an RMC (COMFLTFORCOM, 2009). There is very little 
guidance as to when and under what circumstances the RMC will request ISEA support, 
but typically it follows an onsite visit which failed to return the system to operational 
status. The ISEA will provide support via both distance and onsite means utilizing their 
and the OEM’s engineers and technicians. Once tasked, the ISEA 1s the last line of 


defense, and must return the system to an operational configuration. 


Once requested for support by the RMC, the ISEA begins assisting the 
operational platform and records the effort in their REMEDY data repository. Trouble 
tickets record the details of each instance of a technical assist and populate the REMEDY 
data repository. Throughout the lifecycle of an ISEA technical assist, the trouble ticket 
has three phases, when the ticket 1s opened, closed, and updated. Unlike the operational 
site’s 4790/2Ks and the RMC’s 4790/2Ks, neither the trouble tickets nor the REMEDY 


data repository populate a central site. 


As a result of not being synchronized to a central site, the ISEA’s trouble tickets 
are not used to document materiel status to the operational platform’s chain of command. 
Rather, the trouble tickets document each assistance event to fulfill two objectives. The 
first, especially in a tight budget environment, is to document value to the operational 
forces. By diligently recording each technical assist, the ISEA can precisely demonstrate 
their value when facing budget reductions. This motivation can be seen in Figures 4 and 
5, which depict the forms for opening and closing, respectively, of trouble tickets. 
However, the second objective is gain technical information about the failure event. This 


motivation is, again, seen in Figures 4 and 5, but is more evident in Figure 6, which 


3 Remedy is the name of the software product developed by BMC Software and used by the ISEA. 
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depicts the work log to be completed by the engineer or technician supporting the 
operational platform in problem resolution. Through the work log, all the steps taken with 
ISEA support can be recorded. This includes the corrective action steps, as well as, the 
steps taken to diagnose the failure’s cause. This recording of technical information about 
the failure event is a rich data opportunity, providing details about failures and the 


opportunities to educate ISEA technicians and engineers on their resolution. 


Figure 4: Opening an ISEA Trouble Ticket 
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Figure 5: Closing an ISEA Trouble Ticket 
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Figure 6: ISEA Trouble Ticket Activity Log 
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Table 3 outlines the full set of recorded data opportunities available when 
resolving MILSATCOM failures at operational sites. From the procedure depicted in 
Figure 2, it is possible for a failure event to realize any or all of these data opportunities. 
It is rare for a failure to realize all data opportunities as these represent the most difficult 
system problems, those that require all three support layers, System Maintainer, RMC, 
and ISEA to resolve. It is, however, unknown how many failure events fail to realize any 


of these data opportunities, as their existence is not documented. 
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Table 3: Total Recorded Data Opportunities from Failure Events 


Ship 4790/2K Central Site | Deferred maintenance and material 
Status 

Ship 4790/2L Local Optional update and description of 
corrective action 


Ship 4790/CK Central Site | Records change in shipboard or 
system configuration. 

RMC 4790/2K Central Site | Documents change in configuration 
to a system or deployed system. 

ISEA Trouble Ticket | Local Number of failures requiring expert 
assistance and details about failure 
diagnosis and corrective action. 





C. REPORTING A FAILURE ONBOARD A U.S. NAVY PLATFORM 


In addition to the data opportunities created by the failure resolution activities, the 
processes planned to be inherited by the NMT program provides one additional data 
opportunity. The Casualty Report, or CASREP, is used to report significant equipment 
casualties within the Navy establishment (NSG, 2003). A significant equipment casualty 
is one that cannot be corrected within 48 hours and reduces the platform’s ability to 
perform its primary or secondary mission (NSG, 2003). In addition, there are various 
levels of CASREP priority, levels 2 (lowest) through 4 (highest), based upon the severity 
of the impairment to primary and secondary missions (NSG, 2003). A platform sends a 
CASREP as an official Naval message to a broad distribution of stakeholders resulting in 
operational commanders and support personnel being advised of the status of significant 
equipment malfunctions that may result in the degradation of a unit’s readiness. Further, 


the CASREP message is automatically entered into the Navy Status of Forces (NSOF) 
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database at each fleet commander-in-chief’s site and corrected messages are forwarded to 


the Chief of Naval Operations’ database (NSG, 2003). 


At a high level, the purpose of a CASREP 1s very similar to that of a 4790/2K, 
namely to report the operational site’s material status, however a CASREP has a much 
higher sense of urgency. An operational site should complete a 4790/2K anytime there is 
deferred maintenance, but a CASREP will be completed to amplify and add urgency to 
the delayed maintenance when the operational site is performing an operational mission 
or has plans to perform an operation mission in the near future. While the CASREP will 
provide a request for technical assistance and a description of the problem, given the 
broad and senior distribution of the message, a CASREP can have powerful political 
implications. For this reason, CASREPs receive much more attention than 4790/2Ks or 
REMEDY trouble tickets. As a general rule, a Program Manager will review CASREPs 
on a weekly basis, despite the fact that resolution of a CASREP follows the same process 
as any corrective action event. Namely, the operational site will request assistance from 
the RMC, typically a foregone conclusion when a CASREP is sent, but the ISEA must 
wait to engage until tasked by the RMC. It 1s not uncommon for the Program Manager 
and the ISEA to expend time and effort watching the corrective action process unfold via 
CASREP messages while waiting for, and sometimes soliciting, tasking from the RMC. 
As the old saying goes, “the squeaky wheel gets the grease,’ the operational platform with 
an open CASREP is the squeaky wheel and the focus is making it quiet. 


D. SYNTHESIS OF INFORMATION TO DETERMINE MILSATCOM 
PERFORMANCE 


There are a number of data opportunities associated with resolving a 
MILSATCOM deficiency, but data repositories alone provide little value. Without data 
analysis, it is impossible to determine overall system performance. For instance, 
components of MILSATCOM systems will fail, but it is extremely difficult to determine 
if an isolated event is simply a ‘fact of life’ anticipated failure, or part of a broader, more 
severe issue. Understanding this difference is of critical importance in ensuring 


communication services are available to operational users. 
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The AN/USC-38(V) and AN/WSC-6(V) programs have tasked NAVSEA Surface 
Warfare Center (NSWC) Corona to develop quarterly Reliability, Maintenance and 
Availability (RMA) reports based upon the synthesis and analysis of corrective action 
data. As can be seen in Figure 7, which depicts the data pulls for producing the quarterly 
RMA reports, NSWC reviews the 4790/2Ks, CASREPS, and REMEDY Trouble Tickets. 


Figure 7: Quarterly MILSATCOM RMA Report 
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By analyzing this data, NSWC’s quarterly reports give the Program Manager 
insight into system performance. As can been seen in Figure 8, a quarterly summary 
report for the AN/USC-38(V) shipboard system, the key metrics of Ao (availability), and 
Mean Time Between Failure (MTBF) are reported. In addition, the report identifies the 
system components driving the period’s reliability; for the AN/USC-38(V) report 
depicted in Figure 8, the Internal Data Assembly (IDA) Fiber Optic Gyro (FOG) and 
Solid State Amplifier (SSA) had the greatest impact. Also of note, is the emphasis the 
report places on CASREPs. There are four instances on this single page report of 
CASREP associated metrics; the reasons for this are twofold. First, this is, again, a 
reflection the CASREP’s political weight, making it a must watch item for a Program 
Manager. Secondly, despite potentially only recording a subset of the system 


failures/problems, the CASREP ends up being the best data source. As Figure 7 depicts, 


22 


NSWC reviews 4790/2Ks and attempts to extract as much data as possible, but frequently 
they provide little value. Too often, the 4790/2Ks identify a problem with the system, but 
fail to provide any data about the nature of the failure or document the true status of the 
system. Given the high overlap between the 4790/2K and the CASREP, why is there 
such a discrepancy in the quality? The speculation is that the audience and political 
nature of the CASREP drive in quality. The ISEA generated REMEDY Trouble Tickets 
overlap with the CASREPs, meaning a failure that required ISEA assistance is resolving 
almost always had an associated CASREP, as a result the Trouble Tickets contribute very 
few unique events to the analysis. However, the Trouble Tickets do provide amplifying 
information supporting the Program Manager’s efforts to identify and understand the core 
issue. In summary, a highly political mechanism designed to report the failures with the 
greatest impact and/or most challenging corrective actions currently make up the majority 


of the performance assessments for MILSTACOM systems. 


Figure 8: Sample AN\USC-38(V) NSWC Quarterly Report 
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Hl. CURRENT DATA COLLECTION METHODS ON 
INCREASING SYSTEM AVAILABILITY 


A. SYSTEM AVAILABILITY 


What is system availability? Textbooks define Operational Availability (Ao) as 
the probability a system will be ready to perform its specified function, in its specified 
and intended operational environment, when called for at a random point in time 
(OPNAV 2003). Or, more simply, availability has been described as the percentage of 
time the system will be ready to perform satisfactorily in its intended operational 
environment (OPNAV 2003). Availability is a relevant metric to all aspects of life. For 
instance, mornings are planned and scheduled around the availability of a number of 
systems, including the automobile. The morning schedule utilizing an automobile with 
an availability of 25% is much different than one with 98% availability. The same 
calculations and risks an individual makes based upon their transportation to and from 


work must be made in accomplishing military objectives. 


As the percentage of time a system will be ready to perform satisfactorily in its 

environment, Ao can be simply described by equation I: 
Equation 1 
Ao = Up Time/ (Up Time + Down Time) 

While this straight forward equation conveys the intent of the metric, this form is 
a trailing indicator without any predictive capability. In addition, and more troubling, this 
expression of Ao doesn’t provide any insight as to improving a system when Ao 
performance is inadequate (OPNAV 2003). Rather, the following expressions provides 
both predictive and, more importantly for this analysis, a means to identify and address 
deficiencies in Ao performance. 


Equation2 


Ao = MTBF/(MTBF + MDT) 
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Mean Time Between Failure (MTBF), as the name implies, is a measure of the 
mean operating time between successive system failures; this is a measure of system 
reliability. Mean Down Time (MDT) is the mean time the system is inoperable per failure 
incident (OPNAV 2003). As described by equation 3, MDT is the sum of two attributes 
associated with system downtime, Mean Time To Repair (MTTR) and Mean Logistics 
Delay Time (MLDT). MTTR is the average time it takes to repair a system in its 
operating environment, this includes both the time it takes for the System Maintainer to 
diagnose the failure, and the time required to perform the corrective action. MLDT is the 
down time associated with logistics actions, such as the time required to order and take 
receipt of parts. For example, if in correcting a system failure it takes the System 
Maintainer | hour to diagnose the failure, 2 hours to order a new part, 1 day to receive the 
part, and 2 hours to install the part, the MTTR would be 3 hours and the MLDT would be 
26 hours. 


Equation 3 


MDT = MTTR + MLDT 


B. WHAT WILL INCREASE SYSTEM AVAILABILITY 


1: Increase Mean Time between Failure 


As described by equation 2, an increase in MTBF increases system Ao. There are 
two ways to increase MTBF. The first method is to improve system reliability, 1.e., 
decrease the frequency at which the system fails. System reliability is heavily influenced 
by system attributes determined early in the design process, such as the system’s 
architecture, and, to some degree is inherent to a given design. However, while a macro 
change in reliability may be unachievable, the reliability of a system can be improved. 
Through an iterative process of identifying failure root causes, incremental improvements 
may be achieved. Eventually this process could reach a point of diminishing returns, but 
there 1s value in realizing optimal performance by reaching this point. The second method 
for increasing MTBF is to decrease the number of false negatives, 1.e., reduce operator 


error. There is no difference in result if the system actually has a failure or the System 
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Operator thinks the system has a failure; in either condition, the system will fail to 
accomplish its mission. The pervasiveness of operator error is a function of the training 


and experience of the System Operator; simply, more proficient users results 1n less error. 


2. Decrease Mean Time to Repair 


Decreasing MTTR is heavily reliant on System Maintainer proficiency. The faster 
a system can be repaired the sooner it can support tasking. MTTR is a metric broken into 
two constituent parts, the time to diagnose the failure and the time required to perform the 
corrective action. The speed of diagnosis is a function of the System Maintainer’s 
general troubleshooting skill, the experience with the specific system, and the quality of 
the supporting information. Troubleshooting, often both a science and an art, can be 
improved with training and experience, but also varies with individual talent. Like 
Operating experience, working on a system increases the understanding of a system’s 
design, behavior, and failure mechanisms. Lastly, the quality of technical documentation 
and other supporting materials which guide the System Maintainer have a tremendous 
impact on diagnosis timeline. A cliché joke is the father setting out to assemble 


Christmas gifts only to find the instructions are printed in a foreign language. 


Once the System Maintainer has completed failure diagnosis the time to perform 
the corrective action is affected by both the System Maintainer and system design. Just 
like in failure diagnosis, the System Maintainer’s general skill, and experience with the 
system, along with the quality of supporting information affect the speed at which the 
corrective action 1s performed. Also, the speed of repair is impacted by system design. 
Like MTBF, repair steps are heavily influenced by early design decisions, for instance, 
we have all heard stories of automobiles requiring engine removal to change spark plugs. 
But, again, the design can be incrementally improved. Through the process of analysis 


and incremental improvements, the design can be optimized, reducing repair time. 
3. Decrease Mean Logistics Delay Time 


MLDT is a quantification of supportability that 1s defined to include personnel, 


repair at other levels, supply support, transportation, and other logistics delays not 
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attributable to actual hands-on maintenance time (OPNAV 2003). There is an array of 
items affecting MLDT, but typically the greatest is the time associated with receipt of 
spare parts. When the NMT system is installed, the operational site will receive an 
inventory of On Board Repair Parts (OBRPs). These OBRPs is the operational site’s 
supply of spare parts, the composition of which is based upon the system developer’s best 
analysis and predictions. However, despite the best efforts by the system developer, a 
failure affecting a part not in the OPBRP inventory is possible and probable. In these 


instances the operational site must order a replacement part from Navy supply. 


The time associated to fulfill this requisition request from Navy supply is 
dependent upon both the channel delay and the current inventory. Channel delay is the 
time required for the requisition to be completed, received, processed, and shipped. 
Given the global coverage of Navy ships, shipping delay can have a significant impact on 
MLDT. Also affecting MLDT are inventory levels within Navy supply. If the part is 
available on the shelf, the timeline to process the requisition is relatively short. If the part 
is not in inventory, the MLDT may be significantly impacted. There are several potential 
reasons for inventory shortfalls; common ones include procurement back orders and 
batch repairs. In the case of batch repairs, with cost efficiency in mind, Navy supply 
frequently repairs deficient parts in batches, e.g., repair parts will not be repaired until 
there are a certain quantity to achieve economies of scale. This can create instances 
where the Navy supply has an inventory of deficient parts while waiting for the batch size 
to trigger a repair action. Simply, for whatever reason, if there are no parts in inventory, 


the operational site will wait, increasing the MLDT. 


C. ATTRIBUTES OF AN IDEAL DATA COLLECTION SYSTEM FOR 
INCREASING SYSTEM AVAILABILITY 


1. Increasing Mean Time between Failure 


There are two approaches to increasing MTBF: improve system reliability and 
increase user competency. Addressing these approaches requires data analysis and usage 


by different consumers. 
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To improve system reliability, the individual or organization responsible for 
making system modifications must understand the system’s operational performance. 
There are instances where this responsibility lies with a third party, but typically, this 
responsibility lies with the Acquisition Program Office and System Developer. For this 
audience, the ideal data collection system would capture all issues associated with the 
operation of the system. This issue capture would cover the array of events from minor 
nuisances to major failure events; records would be created every time the system failed 
to meet expectations. In addition, the actions taken to correct the issue, both successful 
and unsuccessful, would be recorded. This comprehensive recording is ideal because it 
provides the Acquisition Program Office and System Developer the ability to identify 
pervasive versus anomalous problems, identify developing trends early, and document 
the conditions surrounding a failure. This information will facilitate application of the 
Acquisition Program Office’s limited resources towards correcting problems with impact. 
In addition to problem identification, a high level of information increases the level of 
problem understanding, creating an opportunity to correct deficiencies more efficiently. 
Efficient deficiency correction, in turn, compounds into an opportunity to stretch 
resources further and correct more system problems. Both phenomena accelerate 


optimization of system reliability resulting in higher availability. 


Increasing user competency and, thus, decreasing instances of operator error, 
requires increased awareness by current system operators and improved training for 
future system operators; both goals can be supported by an ideal data collection system. 
During their career the average NMT System Operator will have a relatively narrow 
experience, having only operated NMT systems at the schoolhouse where they were 
trained, and the one at their operational site. This limited exposure shapes the users 
expectation and understanding of the system and its capabilities. Individually these 
user's experiences provide valuable but very finite insight into the operational 
performance of the NMT system. However, the aggregation of these experiences 


provides almost total insight into the operational performance of the NMT system. By 
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having access to all the events associated with the operation of the NMT terminal, an 
individual user is exposed to information and an understanding well beyond their organic 


opportunity. 


This data would amplify the peer-to-peer training currently taking place at Navy 
sites. As previously mentioned, different operational sites may have varying numbers of 
System Users depending on the size of the site, and these users, like any other work 
environment, share their experiences and knowledge through ad hoc interactions. This 
information exchange increases the overall competency of that site’s user corps, but is 
confined to that site. The ideal data collection system would help transcend this 
proximity constraint increasing the knowledge of the entire user community. By 
leveraging the experience of one to the entire community, the competency of System 


Users will increase, reducing the instances and impact of operator error. 


In addition to improving the knowledge of current users, the ideal data collection 
system would facilitate improved training for future users. The community of NMT 
System Users will have fairly homogenous demographic characteristics. For instance, the 
users will be of similar age, have had the same military education experience, and by 
virtue of having the same IT rate designation all received similar marks on the Armed 
Services Vocational Aptitude Battery (ASVAB)*. As a similar demographic group it is 
reasonable to assume there will be common perspectives prior to, during, and after the 
schoolhouse experience. The training curriculum developers are, however, not of the 
same demographic group. Between the curriculum developers and System Users 
typically exists a gap in age and education, therefore, the training curriculum is a 
translation attempt from one demographic to another. Like all education endeavors, this 
translation has historically had varying degrees of success, but is always based upon an 
assumption by the curriculum developers as to which concepts and material are 
important. The ideal data collection system will allow the schoolhouse and the 
curriculum developers to validate or change these assumptions. This information will 


allow update and refinement of the curriculum based upon real experiences and 


4 The ASVAB is the Armed Forces Vocational Aptitude Test. Test results determine (1) whether one qualifies for 
military service and (2) if so, for which military service jobs. 
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challenges experienced by the System Users. The net result of this assumption feedback 


loop is more effective training, creating higher competency users. 
pa Decrease Mean Time to Repair 


Like increasing MTBF, there are two approaches to decreasing MTTR, increasing 
maintainer competency and making meaningful system improvements. Again, these 
approaches require data consumption by two distinct groups, the System Maintainers and 


the System Developers. 


In isolation, System Maintainer’s experiences, like System Users, provide limited 
Opportunities to increase competency, but, in aggregate, their experiences cover the vast 
majority of knowledge about the system. The ideal data collection system captures rich 
information about the failure diagnosis process, including the symptoms experienced, 
rational for selected steps, and the ultimate problem resolution. This data enables peer- 
to-peer training, turning one maintainer’s experience into a community experience. By 
reviewing this data, a System Maintainer is able to proactively increase his or her system 
knowledge, and rapidly address and apply this knowledge should their system experience 
a failure or, in a more reactive fashion, have a repository of documented solutions. 
Regardless of the proactive or reactive posture taken, the end result is ability of System 


Maintainers to rapidly identify and correct system deficiencies, decreasing the MTTR. 


In addition, this effect is compounded when data is used and created by RMC and 
ISEA technicians. Despite a focused expertise on the NMT system, the RMC and ISEA 
technicians know far from everything, and their competency will grow from the 
community experiences just like the rest of the System Maintainers. With increased 
competency, these technicians can more efficiently assist the System Maintainers in 
corrective actions requiring their support. Part of this increased efficiency will be a 
higher success rate with distance support. High distance support success rates, translates 
into lower MTTR times and lower support costs, which facilitates reallocation of 
resources to fixing system deficiencies. In addition, the ideal data collection method 
would record information about the RMC and ISEA technician’s failure diagnosis 


process including the rational for selected steps. Since the RMC and ISEA technicians 
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have more experience and focused system education than the System Maintainers, this 


data facilitates a “guru-to-expert” tutelage not previously achievable on a broad level. 


The ideal data collection system facilitates ongoing peer-to-peer training and 
“guru-to-expert”’ training, but also facilitates improved initial training. Like the System 
User training, the System Maintainer training is based upon an assumption by the 
curriculum developers. These assumptions are made without the benefits of a similar 
demographic perspective, and feedback is highly beneficial. The net result of this 
assumption feedback loop is more effective training, resulting in more competent System 


Maintainers. 


Like the training curriculum, technical manuals suffer from the same 
demographic disparity. Technical manuals are typically written by the System 
Developers whom have a different perspective and understanding than the System 
Maintainers. Again, this disparity is bridged via assumptions made by the System 
Developers. The good assumptions will result in the clear conveyance of information to 
the System Maintainer. The bad assumptions could leave the System Maintainer 
bewildered, prolonging the duration of the repair, or worse, result in further system 
damage, or worse yet, create a safety hazard and injure the System Maintainer. While 
technical manual writers are professionals and there are processes to validate and verify 
the content, iterative feedback on the assumption provides a great opportunity for 


improvement. 


Lastly, the Acquisition Program Office and System Developer can use the 
corrective action data to improve the system maintainability. Like increasing reliability 
through iterative system improvements, the system repair can be made easier. While a 
good designer will take repair actions into consideration during the design process, they 
are always at a context disadvantage. For instance, without being onboard a moving ship, 
in the dark, standing on their tip-toes while reaching around cables, the choice of fine 
versus course connector threads may seem insignificant. With information about these 
experiences and challenges, the Acquisition Program Office and System Developer can 


take steps to improve these unforeseen problems. 
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3. Decrease Mean Logistics Delay Time 


The largest contributor to MLDT is the time associated with shipping and receipt 
of replacement parts and avoiding this process would significantly reduce MLDT. By 
capturing all the issues associated with the system, the Acquisition Program Office and 
the System Developer can identify the truly pervasive system problems. With this 
information the first approach of eliminating problems through design modifications may 
not be feasible. In these instances, there is an opportunity to augment the operational 
site’s spares compliment with the frequently failing components. In addition to 
identifying system components, an ideal data collection system would identify issues 
with corrupted or frequently lost computer files facilitating a modification to operational 
procedures for data archiving. In these instances, an ideal data collection system 


facilitates avoidance of the largest contributor to MLDT, channel delay. 
4. Summary of an Ideal Data Collection Process 


An ideal failure data collection system provides an opportunity to address and 
improve all components of Ao. As enumerated by Table 4, the ideal data collection 
system captures all system anomalies and the steps taken to resolve them. Once 
collected, this information is consumed by different stakeholders to improve their role in 


ensuring the NMT provides the required NMT capability. 
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Table 4: Ideal Data Collection Summary 


Item # Data Recorded Data Customer Purpose Ao Parameter 


Affected 


Acquisition 
Program Office 
All system anomalies System Improvement MTBF 
and System 
Developer 
ia ae Training Curriculum 
All system anomalies e and Technical MTBF 
and System 7.5. 
Publication Updates 
Developer 
Acquisition 
Ail ssien snomalike Program Office OBRP requirement MLDT 
and System updates 
Developer 


Community Training 


System 
System Maintainer Failure Maintainer Bee 
5 Diagnosis and Corrective Community and Trainin MTTR 
Action RMC and ISEA e 
Technicians 








System 
RMC and ISEA Technician Maintainer Guru-to-Expert 
Failure Diagnosis and Community and _ MTTR 
Corrective Action RMC and ISEA Training 
Technicians 
Acquisition 
Failure Diagnosis and Program Office 
Corrective Action and System ee cen Neos 
Developer 
pease Training Curriculum 
Failure Diagnosis and Program Office C 
; and Technical MTTR 
Corrective Action and System Soe 
Publication Updates 
Developer 


Acquisition 
Program Office 
and System 
Developer 


RMC and ISEA Technician 
Failure Diagnosis and 
Corrective Action 


Training Curriculum 
and Technical MTTR 
Publication Updates 
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D. EVALUATION OF THE CURRENT DATA COLLECTION METHODS 
ON INCREASING SYSTEM AVAILABILITY 


The NMT system is scheduled to inherit the problem resolution process and data 
recording process currently employed by the AN/USC-38(V) and AN/WSC-6(V) systems 
described in Chapter II. The problem resolution process and data recording process seeks 
to satisfy two goals, both pertaining to readiness. The first goal has a very short time 
perspective and is concerned with documenting and reporting the current readiness of a 
platform. This information is, obviously, very important and relevant when an 
operational site is executing an active mission, or preparing to execute a mission in the 
near future. If an NMT experiences a failure that precludes, impacts, or increases the risk 
for an operational platform in performing its mission, this information needs to be made 
available rapidly to the command chain. With mechanisms such as the CASREP, the 
existing system satisfies this goal very well. However, there is a second goal, one that 
takes the longer view of improving system performance, its users, and its support 


structures; this is the area being evaluated. 


In the previous section, attributes of an ideal system were proposed as means 
against which to evaluate the current system and any proposed modifications. The ideal 
attributes may not be achievable in execution, so a perfect score for any system against 
any attribute is unlikely, but it does allow a comparison of several different systems using 
the ideal as an intermediary. There were nine of these ideal attributes identified that 


9 


support the “longer view” of increasing system Ao. The sections below evaluate the 


current system against these nine attributes. 


Table 5: Ideal Attribute 1 


Item # Data Recorded Data Customer Purpose Ao Parameter 
Affected 


Acquisition Program 
All system anomalies Office and System System Improvement 


Developer 
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The first attribute of the ideal system is to record information about all system 
anomalies for the purpose of improving the system. The current system collects 
information using the 4790/2K, CASREP, and REMEDY Troubles Tickets, but this 
information is neither as comprehensive nor as detailed as the ideal. Currently, when the 
System User experiences an anomaly they must make a decision about the need to alert 
the System Maintainer. There are a number of failures modes, predominantly software in 
nature, which the System User may be able to correct, such as reloading a corrupted file, 
restarting program, or restarting the entire system. Each one of these events could result 
in a loss of communications for several to many minutes, but could occur and be resolved 
without any documentation. Without being documented, it is possible the organizations 
with the ability to correct them, the Acquisition Program Office and System Developer 
may not know they exist. The same threshold to reporting problem exists once the 
System Maintainer is notified of an issue. The System Maintainer can address software 
issues, but can also correct hardware deficiencies that may not trigger the completion of a 
4790/2K. This creates the opportunity for a class of failures that impact or temporarily 
impair communications that, because they are not known, could exist throughout the life 


cycle of the system. 


In addition, for anomalies that are documented, the documentation often fails to 
include details that would be beneficial in the development of a correction to the 
deficiency. For instance, the System User experiences the failure, yet the User does not 
complete any of the failure documentation. Once the System Maintainer initiates a 
4790/2K, potentially lost 1s contextual information about what the System User was 
trying to accomplish and expecting. This information can help the System Developer 
understand and fix the deficiency in a number of ways, one of which is to efficiently 
recreate the event. Further, whether the System Maintainer chooses to complete a 
4790/2K or a CASREP or not, the primary purpose of the documentation is readiness 
reporting, not long-term improvement. With this in mind, the records focus on what this 
system cannot do, with little detail on the actual failure mode. The same is true of the 
RMC-generated 4790/2K, as the bulk of the dialogue used to describe and understand the 


failure is done via perishable means such as voice communication or ad hoc methods 
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such as chat or e-mail. In their REMEDY Trouble Tickets, the ISEA attempts to record 
richer information about the details and nature of the failure, but the ISEA sees only the 
worse conditions, and the REMEDY Trouble Tickets do not reconstruct the data lost by 


other stakeholders in the failure resolution process. 


Table 6: Ideal Attribute 2 


Item Data Recorded Data Customer Purpose Ao Parameter 
Affected 


Acquisition Program Training Curriculum and 
All system anomalies Office and System Technical Publication 


Developer Updates 





The second attribute of the ideal system is to record information about all system 
anomalies for the purpose of updating the training curriculum and technical publications 
with the goal of improving System User competency. Again, the current data recording 
system is hampered by both the threshold trigger for reporting and the fact that the 
System User does not contribute any data to the process. Without either all the instances 
or the input of the actual user, it is very difficult to assess the frequency and 
pervasiveness of operator error. With little information, the technical publications and 
the training curriculum typically receive littke to no improvement from the failure 
process. Typically, the only time these documents get addressed throughout the failure 
process is if they are explicitly stated in a CASREP. Statements concerning these 
documents’ specific areas for improvement are rare, but rather often read like a statement 
of frustration by the operational site, as if to say, “the system isn’t working and we can’t 
fix it as a result of insufficient training and poor technical manuals.” With the CASREP 
being a highly political document, these statements may invoke a reactionary look into 


the publications, but, without specific information, typically yield little improvement. 


oy, 


Table 7: Ideal Attribute 3 


Data Recorded Data Customer Purpose Ao Parameter 
Affected 


Acquisition 


Program Office OBRP requirement 


All system anomalies 


and System updates 


Developer 





The third attribute of the ideal system is to record information about all system 
anomalies for the purpose of updating the operational sites’ OBRPs complement. 
Whether via completion of a 4790/2K, 4790/CK, or CASREP, the current system does an 
effective job capturing the demand for replacement parts, allowing ongoing calculation 
and analysis of the operational site’s spares inventory. There is potential that a failure 
could be corrected with an existing OBRP without being recorded. This situation is not 
desirable, however, when trying to evaluate the spares complement, understanding 
needed parts not currently in the spares compliment is more important than knowing the 
utility of the parts currently in the spares compliment. This is because the existing 
complement represents a sunk cost, 1.e., they have already been bought, where as the 
Acquisition Program Office must apply its limited resources to add a spare to the 
inventory. The 4790/2K and perhaps 4790/CK and CASREP allows the Acquisition 
Program Office to evaluate the opportunities to improve availability by augmenting the 


OBRP package. 


Table 8: Ideal Attribute 4 


Item Data Recorded Data Customer Purpose Ao Parameter 
Fi Affected 


System User _ 
All system anomalies Peer-to-Peer Training 
Community 
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The fourth attribute of the ideal system is to record information about all system 
anomalies for facilitating peer-to-peer training by leveraging the experiences of 
individual users to that of the entire community. Used for this approach, the ideal system, 
increases user competency and reduces the frequency of operator error by allowing other 
operators to learn more about the capabilities and limitations of the system through the 
experiences of their colleagues. The current data collection system fails to address this 
attribute. None of the data produced, whether in a 4790/2K, 4790/CK, CASREP or 
REMEDY Trouble Ticket, is shared or available to other System Users. There is 
opportunity for peer-to-peer education within an operational site as a result of geographic 


proximity, but this is where the opportunity ends. 


Table 9: Ideal Attribute 5 


Item Data Recorded Data Customer Purpose Ao Parameter 
Affected 


System Maintainer 


System Maintainer Failure Community and 


Diagnosis and Corrective Action RMC and ISEA 


Peer- to-Peer Training 


Technicians 





The fifth attribute of the ideal system records information about the steps taken by 
the System Maintainer when diagnosing a failure and performing the corrective action. 
Collecting and sharing this information amongst the community allows other System 
Maintainers to learn from the each other’s successes and failures. With the current data 
recording system, the System Maintainers documents very little about the corrective 
action process and nothing about the diagnosis process. For instance, with a 4790/2K a 
System Maintainer could state replacing a certain part corrected the deficiency, but 
cannot record any information about performing the repair. For challenging repairs, this 
information would be beneficial to all System Maintainers, reducing the time and 
probability of collateral damage to other components during system repair. The System 
Maintainer can record this type of information in a 4790/2L, but, per direction, the 


system does not support reporting this information beyond the operational site. So, like 


39 


the 4790/2K, the information contained in the 4790/2L isn’t shared to other System 
Maintainers, but unlike the 4790/2K, the information is not distributed to anyone; no one 


beyond the operational site sees the information. 


With regards to information on the diagnosis process, the current system does not 
record any information. Again, the 4790/2K may record, at a high level, the corrective 
action taken, but it does not record any information about how the System Maintainer 
arrived to that conclusion or how many red herrings were chased until the root cause was 


identified. 


In summary, the current data recording system does not support peer-to-peer 


training of System Maintainers. 


Table 10: Ideal Attribute 6 


Item Data Recorded Data Customer Purpose Ao Parameter 
+ Affected 


System Maintainer 
RMC and ISEA Technician 


Community and Guru-to-Expert 


RMC and ISEA Training 


Failure Diagnosis and Corrective 


Action _ 
Technicians 





The ideal data collection system records information to support another type of 
training, that being RMC or ISEA technicians training System Maintainers, or ““Guru-to- 
Expert” training. The opportunity is similar to that of peer-to-peer training, by 
documenting both the diagnosis and corrective actions taken by the RMC or ISEA 
technicians, the entire community of System Maintainers can learn from the knowledge 
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and expertise of these system “Gurus.” Under the current data recording system, RMC 
technicians document all information via 4790/2K. Whether used by the System 
Maintainer or the RMC technician, the 4790/2K is hampered by the same constraints of 
limited corrective action information, no diagnosis information, and no visibility by the 
rest of the System Maintainer community. With the current system, the “Guru-to-Expert”’ 


training is performed on an individual basis between the assisting technician and the 
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System Maintainer requiring assistance. ISEA technicians record a richer set of data, 
documenting both the steps taken and the rational for doing in the log portion of the 
REMEDY Trouble Tickets. This information is reviewed by other ISEA technicians 
allowing knowledge to diffuse throughout the ISEA technicians, but this information is 


not shared beyond this small group. 


Table 11: Ideal Attributes 7 and 8 


Data Recorded Data Customer Purpose Ao Parameter 


Affected 


Acquisition 
Failure Diagnosis and Corrective Program Office 
System Improvement 
Action and System 
Developer 


Acquisition _ 
Training Curriculum 


Failure Diagnosis and Corrective Program Office 


and Technical 
Action and System 
Publication Updates 
Developer 





In addition to facilitating peer-to-peer training, the collection of failure diagnosis 
and corrective action facilitates improvements to the system and its supporting 
infrastructure. As previously noted, seemingly insignificant items, such as thread choice, 
can have a disproportionate impact on the difficulty and time associated with performing 
a corrective action. With information, the Acquisition Program Office and System 
Developer may be able to eliminate these challenges. The same is true of training 
curriculum and technical publications; they too can be improved once the challenges are 
understood. With the current data collection system, the Acquisition Program Office and 
System Developer may be able to infer some changes to the training materials and 
technical publications based upon the types of failures experienced. For example, if a part 
in the NMT experiences frequent failures, and attempts at improving the part were 
unsuccessful, enough information could be gained to warrant changes to the training and 


technical publications notifying the System Maintainer to look at one particular 
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component first. However, through limited information in the 4790/2K and lack of 
distribution of the 4790/2L the current system doesn’t support fine tuning the repair 
process. Lastly, much like instances of System User error, training and technical 
publications may be mentioned in CASREPS, but statements concerning these documents 


are very general, often reading like a statement of frustration. 


Table 12: Ideal Attribute 9 


Item Data Recorded Data Customer Purpose Ao Parameter 
+ Affected 


Acquisition 


RMC and ISEA Technician Training Curriculum 
Program Office 
Failure Diagnosis and Corrective and Technical 
and System 
Action Publication Updates 
Developer 





The current data collection system only uses limited information from RMC and 
ISEA assistance action to improve training curriculum and technical publications. The 
information provided in RMC technician generated 4790/2Ks provides limited 
information for improving either the maintainability of the system or technical 
publications. However, the Acquisition Program Office and System Developer does use 
the data recorded in the log portion of the Remedy Trouble Ticket to identify and address 
issues with the technical documentation. Typically these issues are technical errors or 
omissions. Training curriculum typically does not reap as much of a benefit from this 


data, but the training curriculum could be improved if an egregious error was discovered. 


Table 13 is a summary of the current data recording systems performance against 
the ideal recorded as a score. The score was calculated on a scale of | through 5. A 
score of 5 1s equivalent to the ideal, while a | represents a failure to address the attribute. 


Scores of 2 through 4 represent document increased performance relative to ideal. 
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Table 13: 


Data Recorded 


All system anomalies 


All system anomalies 


All system anomalies 


All system anomalies 


System Maintainer 


Failure Diagnosis and 


Corrective Action 


RMC and ISEA 
Technician Failure 
Diagnosis and 


Corrective Action 


Data 
Customer 


Acquisition 
Program 
Office and 
System 


Developer 


Acquisition 
Program 
Office and 
System 


Developer 


Acquisition 
Program 
Office and 
System 


Developer 


System User 


Community 


System 
Maintainer 
Community 
and RMC 
and ISEA 


Technicians 


System 
Maintainer 
Community 
and RMC 
and ISEA 


Technicians 


Purpose 


System 


Improvement 


Training 
Curriculum 
and Technical 
Publication 


Updates 


OBRP 
requirement 


updates 


Peer-to-Peer 


Training 


Peer- to-Peer 


Training 


Guru-to-Expert 


Training 





Summary Score of Current vs. Ideal System 


Ao 


Score 
Parameter 
Affected 


MTBF 


MTBF 
MLDT 


MTBF 


MTTR 
MTTR 





Data Recorded 


Failure Diagnosis and 


Corrective Action 


RMC and ISEA 


Technician Failure 
Diagnosis and 


Corrective Action 


Failure Diagnosis and 
Corrective Action 


Data 
Customer 


Acquisition 
Program 
Office and 
System 


Developer 


Acquisition 
Program 
Office and 
System 


Developer 


Acquisition 
Program 
Office and 
System 


Developer 
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Purpose 
Parameter 
Affected 


System 


Improvement 


Training 
Curriculum 
and Technical 
Publication 


Updates 


Training 
Curriculum 
and Technical 
Publication 


Updates 





IV. ONLINE COMMUNITIES 


A. WEB 2.0 AND ONLINE COLLABORATIVE COMMUNITIES 


To many, the word community may elicit thoughts of their neighborhood or even 
their city. However, Web-based online technologies are facilitating new community 
models which break these traditional geographic constraints. These tools are 
accomplishing this through an unparalleled level of participation in content generation. 
Users are able to develop and share content around an array of topics facilitating the 
formation of communities. In the “Web 2.0” construct, as the application of these 
capabilities is coined, every user is not only a consumer of data, but also a data producer 
and data organizer (Sankar & Bouchard, 2009). The moniker Web 2.0 conveys this 
participative difference from other uses of the Web technologies coined “Web 1.0,” often 
disparaged as “brochure ware’; a term that highlights the unidirectional flow of 
information from publisher to subscriber. The Web 1.0 model of concentrated publishers 
providing information to the masses runs contrary to Web 2.0’s fundamental paradigm of 


participation. 


Despite the differences in use paradigms, the underlying technologies are very 
similar; there is not one technology driving the deployment of Web 2.0 (Sankar & 
Bouchard, 2009). Web 2.0 employs the same technologies such as HTML, XML, Java, 
and Java Script used in previous Web applications. Some extrapolations of these 
fundamental tools, such as Asynchronous Java and XML (AJAX), which support ad hoc 
high frequency updates to portions of the user’s screen and improve usability of these 
capabilities, are supporting and enabling functions rather than causal technologies. Web 
2.0 has also benefited from macro computing trends of increased access to bandwidth, 
increased computing capabilities and cheap storage. But, again, these effects are enablers 
vice creators. Fundamental to Web 2.0 is the philosophy that a mass of individuals can do 
better job of creating content than a small cadre of individuals. This participative 


paradigm is executed through via several Web 2.0 models 
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B. WEB 2.0 MODELS 


1. Web Logs 


Web logs, more commonly known by their contracted name, blogs, are Web- 
based tools that enable people to share and discuss information (Sankar & Bouchard, 
2009). The blog owner initiates the discussion through a blog posting. A posting can be a 
written statement, a picture, a video, or a combination of all these elements, really 
anything the owner wishes to share. Users participate in blogs by reading the owner’s 
postings, and contributing their thoughts, opinions, questions, or whatever they desire, as 
a comment. Also, the users can post comments in response to other user’s comments 
resulting in discussions between the owner and users, as well as, among the users 
themselves. These dialogues are recorded allowing participants to contribute within their 
time constraints, 1.e., participation is not required at a specific time, making blogs a 
relatively time insensitive method of collaboration. Recording of these postings and 
discussions facilitate other users passively participating in the conversation and create a 


reference archive of these dialogues. 


Online diaries are a frequent connotation for blogs, but blogs are very pervasive 
sources of information with radical implications for traditional information sources. In 
the United States an estimated 26.4 million users have started a blog, of those that are 
active, 46% characterize themselves as professional bloggers (Sankar & Bouchard, 
2009). These professionals typically write about a specific area such as technology, 
economics, or pop culture—treally, the topics are endless—but one area significantly 
affected by blogs has been news reporting. For many news consumers blogs represent 
the preferred medium for information on current events and important issues. This trend 
has greatly affected the traditional media companies, especially newspapers, which have 
experienced enormous changes over the past couple of years. The first White Houe Press 
credentials issued to a dedicated blogger were given in March 2005, a symbolic inflection 
point in the newspaper business with almost 95% of the top newspapers now having 
reporter blogs and several newspapers closing or converting to an all digital format 
(Sankar & Bouchard, 2009). 
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In addition to changing industries, blogs have been embraced by companies as a 
means to inform customers and capture their opinions. Companies use blogs to obtain 
customer feedback about specific products or general initiatives. For instance, Nike uses 
their blog to debut new products, while General Electric uses theirs as a news reporting 
service to discuss their latest technical innovations. Both companies receive input from 
their customers; Nike is able to get explicit feedback about the likes and dislikes on a 


specific product, while GE can assess enthusiasm for a particular area of research. 
2 Wiki 


The wiki model allows users to co-create and collaboratively develop content. By 
contrast to blogs where the chronological order of owner postings and user comments 
facilitate a conversation, a wiki allows all users to create or edit content placed within the 
wiki page. For instance, as a simple example, driving directions posted to a wiki site by 
an individual could be modified or updated by another to correct errors, inform of road 
construction or even an accident. This mass editing function facilitates two phenomena; 
the first is short publishing cycles and, the second, a convergence toward quality. Users 
are able to update wiki content on a constant basis,9° ensuring the most current 
information is available to wiki users via incremental updates, rather than waiting for a 
specific or threshold to initiate a new revision. In addition, these updates performed by a 


community of users increases the content quality. 


Based on an extrapolation of the adage that ‘two heads are better than one,’ a 
community of individuals reading and editing content are able to create a product of 
comparable or superior quality in an efficient manner. In 2005 “Nature” magazine 
conducted a study to determine the relative accuracy of Wikipedia, a wiki based 
encyclopedia, and the Encyclopedia Britannica on science topics. Forty two topics were 
analyzed, for which Wikipedia averaged 4 errors per topic while Encyclopedia Britannica 
averaged 3. With this one additional error per article “Nature” found Wikipedia to be 


comparable to Encyclopedia Britannica for science topics (Giles, 2005). Wikipedia’s 


> Wiki applications provide features that prevent users from editing information at the time. If one 
user 18 performing edits to the information others are prevented from editing until the other users have 
finished. 
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performance as an information source is even more impressive when the fact that in 
addition to the 42 topics analyzed Wikipedia has amassed 2.8 million more articles than 
Encyclopedia Britannica. In addition the Wikipedia community is able to immediately 
correct errors making them available to all users in a matter of minutes, whereas, 


Encyclopedia Britannica must publish addendum sheets or wait for a new volume. 


While Wikipedia is the most famous application, many companies and 
organizations are using wiki’s for collaboration within companies and with their 
customers. As a corporate user, Sun Microsystems had over 600 employees and affiliates 
using their wiki for collaboratively creating documentation after one month of use 
(Mader, 2008). The National Constitution Center, a non-profit for increasing public 
understanding of the constitution, uses a wiki to coordinate and collaborate on 
information between constitutional scholars and the general public; an annual community 
of 165,000 users (Mader, 2008). As a last example, fans of the Scottish Football Club are 
using a wiki to create the definitive history of the club in an extraordinarily rich fashion. 
This history includes all the traditional items such as the team’s chronology and 
significant events, but, by using the wiki, it also records the fan’s experiences and 
feelings associated with the club. The fans have created over 3,500 pages of content, 
added 7,000 images and made over 18,000 contributions, resulting in a team anthology of 


breadth and essence unachievable through traditional means (Mader, 2008). 


3. Folksonomy 


The challenge with any filing system, whether physical or electronic, is making a 
decision about where to place something. Once an item 1s placed in a location, the ability 
for it to be found again is based upon the seeker having similar opinions about where it 
belongs. As a simple example, couples in a domestic setting may have different opinions 
about where keys should be stored, one believing they should be kept in plain view will 
invariably be searching, and probably frustrated, when seeking them if they were last 
used by a spouse who believes they belong in a drawer. Clearly this problem increases 


dramatically as the number of seekers and number of items being sought increases. 
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Metadata, data about data, was designed to combat this problem by allowing systems to 


collocate information and help users find relevant information. 


Metadata has historically been assigned by professionals, such as in library 
catalogues, resulting in two significant limitations. First, by relying on a cadre of 
professionals there is a capacity constraint as to how much data can be catalogued. This 
problem is compounded by the rapid rate at which data is being created in the Web 2.0 
paradigm; there is simply no way a group of cataloguers can keep pace with the rate of 
creation. Secondly, albeit professional catalogers use defined taxonomies, the seeker is 
still removed from the process and the potential for differences of opinions and its 
associated problems remain. One-way to combat the first problem is having the author 
catalogue their data. This eliminates the capacity constraint, but compounds the second 
problem because most authors lack the professional training to apply taxonomies and 
have definite opinions about the associations of their work. Web 2.0 allows both these 


challenges to be addressed through folksonomy. 


Folksonomy, a hybrid of folk and taxonomy, is an organic system of organization 
based upon tags placed by users (Pink, 2005). Tags support both data descriptions and 
scoring. Instead of placing electronic files into folders which, because a file can only 
exist in one folder is restrictive, tags are without hierarchy allowing for many different 
categorizations and associations (Mader, 2008). For example, a picture of skydiver could 
be valuable to one author writing an article titled “Life’s Great Adventures” and another 
one writing an article titled “Stupid Things Not To Do.” With tags this picture could be 
described as ‘adventure,’ ‘adrenaline rush,’ and ‘fun’ to allow one author to find it, while 
also being described as ‘risky,’ ‘dangerous,’ and ‘death’ allowing the other to find it. In 
addition to describing the data, tags allow for users to evaluate or score data. There are 
different scoring mechanisms ranging from counting the number of instances the data 
was accessed to actual grades. The more complex the scoring mechanism the more 
subjective the scores and even the attribute being scored. For instance, graded scoring 


systems typically ask the user to rate the data, it does not specify if the rating 1s an 
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evaluation of quality, personal interest, or some other attribute. Regardless, this scoring 
mechanism and descriptive tagging have supported many of the archetype Web 2.0 


capabilities. 


YouTube, Flickr, and Digg are three capabilities that have become synonymous 
with Web 2.0. YouTube allows users to upload and share videos they have generated 
with anyone and everyone, an activity that has proven very popular with over 100 million 
videos available on YouTube. As producers post and users view videos, they are 
described and scored via tags. When searching for a video, users can search by a 
descriptive term or view videos based upon their popularity. Flickr allows a similar 
service as YouTube, but with photographs instead of motion video. Photographers post 
their digital photographs, currently in excess of 3 billion, and others are able to search 
through them based upon descriptive terms and scores. In contrast to YouTube and 
Flickr, which focus on a specific data format, Digg allows users to discover and share all 
content across the Web. This includes videos, pictures, audio tracks, and written word, 
essentially, if it is available on the Web it can be tagged and used with Digg. The Digg 
tags can be descriptive, but the focus is heavily on scoring, which facilitates a Web 
popularity contest. As Web users find content, they score it with a Digg tag. Based upon 
how and how many users have scored the content, it will rise in popularity. The top- 
scoring content is placed on the Digg homepage, similar to the newspaper practice of 


putting the most important news on the front page. 


As the examples of YouTube, Flickr, and Digg demonstrate, folksonomy based 
capabilities are very popular in facilitating communities with similar interests but have 
yet to see broad application in communities with specific goals such as corporations. 
YouTube, Flickr, Digg, and others do an excellent job of discovering, associating, and 
scoring content for a community formed around similar interests or hobbies. With these 
tools member of a community interested in home audio can find articles, how-to videos, 
and pictures. But, in these applications the penalty for missing content 1s very low or 
nonexistent, since their value is in bounding the breadth and depth of content, not 
isolating a specific instance. Often, this is contrary to the mission of a corporation, which 


spends resources for the generation and collection of its intellectual capital. With this 
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investment, companies currently see the value in investing in organization schemes 
governed by strict taxonomies. For instance, an electronics engineering firm would be 
uncomfortable with their designs floating in the morass of company data only 
discoverable by descriptive word searching. However, while a folksonomy based primary 
structure 1s unpractical in the corporate setting, as a secondary structure there is an 
opportunity for it to support discovery and reuse by other members within the 
organization. As an example, by using a folksonomy schema the electronics engineering 
firm could interrogate their design repositories to help with finding solutions and design 


reuse when faced with a difficult technical problem or a challenging cost objective 
4. Aggregation 


Aggregation is a mechanism for collecting a presenting content from regularly 
changing Websites such as blogs and news sites (Sankar & Bouchard, 2009). Staying 
current with interesting content via the brute force method, 1.e., frequently checking sites 
for updates in content, rapidly becomes impractical as the number of sources being 
followed increases. The user would either run out of time or get annoyed with the 
process; aggregators solve this problem. An aggregator is an application that collects, 
consolidates, and displays updates from sites the user is interested. The display of the 
update looks very similar to an e-mail inbox, identifying the material’s source, subject 
line or headline, and a short summary or segment of the update. Just like an e-mail 
inbox, the user can rapidly parse through the updates choosing which to ignore and which 
are of interest. Once interested the user follows a Web link provided in the content to the 
site containing the full update. To receive the updates to their aggregator a user 
proclaims their interest in the site’s content by subscribing. For example, a user may 
have an interest in a sports blog about their favorite team, while visiting the blog the user 


can subscribe and will be notified when the owner makes a new posting. 


Aggregation, or subscription feeds, unlike the other Web 2.0 models discussed 
have not resulted in capabilities unto themselves. By this, it is meant there isn’t an 
analog to blogs, Wikipedia, YouTube or Flickr, a capability that pivots on the 


aggregation model. Rather, aggregation is a supporting model used widely, if not 
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ubiquitously, by news sites, blogs, wikis, and other content providers. However, by 
allowing the users to manage and rapidly parse a wider array of content, aggregation 


increases the value of these other content sources. 


There is one emerging exception where aggregation moves from subordinate and 
enabling to the limelight; this exception is Twitter. Twitter is a service that allows users 
to receive notes, or micro-blogs, via subscriptions. However, Twitter is not a traditional 
aggregator. Users can only subscribe to content provided by other Twitter users, whereas, 
a traditional aggregator supports subscriptions to any site which supports a feed. Also, 
with Twitter, the updates are the content, not a headline or summary. Regardless of the 
differences, the subscription and timely content notification features that aggregation 
provides are critical to the Twitter model. With this model, Twitter has a connotation as 
a service allowing teenagers to keep in touch with each other, but has also proven 


beneficial in distributing public service announcements during natural disasters. 


> Mashup 


A mashup is a new Web service created by combining two or more Web services 
(Sankar & Bouchard, 2009). As an example, one Web service might allow users to 
contribute a list of street addresses and another Web service provides mapping services, 
by combining these two, a new service, a tailored map depicting the location of the items 
of interest has been created. This exact process was performed by Starbuck’s coffee 
chain customers to track store closings following a company announcement. From a 
technical perspective, mashups are enabled and heavily dependent upon Application 
Programming Interfaces (APIs). APIs act as an interface description or interface control 
document defining the input requirements along with the output requirements. Defining 
these input and output requirements creates a clear, loosely coupled construct for 
communication between the various services. The developer of a service publishes the 


API definition, which allows others to integrate that service into their application. 


Much like the Starbuck’s example, individuals and corporations have combined 
Web services via APIs to create a seemingly endless number of mashups for personal or 


corporate use. As a personal user, Cleveland Wilson integrated a wireless phone, bar 
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code scanner, and the Amazon.com API into a system allowing him to query selling 
prices for book titles on Amazon.com while visiting garage sales and thrift shops (Hof, 
2003). This has facilitated a $100,000 annual book arbitrage business for Cleveland. In a 
similar fashion, Zillow.com provides participants in the real estate market another, 
broader, source for price estimation. Zillow.com uses data from the county in which the 
real estate resides, such as last sale date, tax assessment, property description and last sale 
price, in combination with the mapping capability provided by Google.com to create a 
map of estimated housing prices. A Zillow.com user types in the address of an interested 
property to receive a price estimate. The price estimate is based upon data specific to the 
interested property and other properties in the area; this creates another data point for the 
critical “comp” when performing real estate transactions. As a last example, 
Salesforce.com, a Customer Relationship Management (CRM) business providing tools 
that allow organizations to document and manage their transactions, discussions, and 
overall interactions with their customers, has published an API and created an application 
development platform that allows system users to generate mashups based upon their 
specific need. Users have created mashups mapping customer locations, developed 
graphics based on sales performance, and many other features and combinations. By 
embracing the mashup model Saleforce.com has, to some extent, eliminated the essential 
question of product development, “what features do the customers value?” With the 
highly modular API mashup model, the customers select applications they value or 


simply develop one of their own. 


From personal user to corporate strategy mashups have had a tremendous impact, 
but they are also the model behind what many will believe will have the largest impact 
yet, social networking. Social networking is another Web 2.0 archetype, which allows 
users to connect with other users for interaction and sharing; what the users share and 
interact about typically defines the social network. The two most popular social 
networking sites MySpace.com and Facebook.com have become, like YouTube and 
Flickr, almost synonymous with Web 2.0. The fundamental application for these sites 1s 
the user profile database. Users populate the database with information about 


themselves, such as name, age, location, hobbies etc., but this database also records 
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linkages to other user profiles creating a Web, or network, of associations between users; 
MySpace.com and Facebook.com call these associations “friends.” This database of 
users and associated “friends” is accessed via an API, which allows any number of 
mashup services to be created. Common applications are photograph and video sharing, 
blog subscription, note writing, schedule coordination, and passing information about 
favorite interests or products. It is the last use that has created the most interest amongst 
corporations and organizations. Via the user database, it is possible to identify individuals 
with the highest number of associations, and advertisers believe these high traffic 
network nodes disproportionately impact the exposure and opinion of all network 
participants, providing tremendous opportunity to distribute product messages. 
Advertisers believe this opportunity 1s relevant for an array of products from laundry 
detergent to political candidates; several analysts have attributed much of Barack 
Obama’s success in being elected President to his use of, and popularity within social 


networks. 


With rapid growth and over 250 million users, Facebook has become the highest 
profile social networking site, but there are other sites. Linkedin.com allows 
professionals to network based upon professional relationships. Librarything.com allows 
readers to share and find books based upon items they have read. Classroom2.0 allows 
users to collaborate and share about the uses of Web 2.0 technologies in the classroom. 
This is only a shortlist of the communities that have emerged using social networking, 


mashups, and, in general, Web 2.0 capabilities. 
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V. SATCOM PERFORMANCE DATA COLLECTION USING 
WEB 2.0 MODELS 


A. CONSIDERATIONS IN DEVELOPING WEB 2.0 COMMUNITIES 


When developing a system to facilitate interactions within a community it is 
critical to identify the community members and their roles within the community. There 
are three roles members can take within their online community: consumer, developer, or 
contributor. Consumers are a role that typically all members of the community occupy, 
meaning all members participate (e.g., read, listen, etc.) in the information other people 
have provided. This can be either a stand-alone role, or coupled with the other two roles; 
in the stand-alone instance, the community member is a passive recipient of information. 
Developers are the community members that determine the topic or focus of the 
community content or discussion. Using a business meeting as an analogy, the developers 
determine the meeting’s agenda. Contributors of the community are those members that 
participate in community interactions once a topic has been developed. Using the meeting 


analogy again, these are the meeting’s active participants. 


With an understanding of the member roles, the proper Web 2.0 models can be 
chosen to support the community. For instance, in examining Web 2.0 models, both Nike 
and the Scottish Football Club were cited as examples, but because of differing 
community member roles different Web 2.0 models were used. In the Nike instance, 
their community consisted of two dominant members, Nike and their customers. In this 
community Nike developed the topic and the users provided feedback through a 
discussion format; both roles consumed information. The disparity between the 
developer/consumer role played by Nike and the contributor/consumer role played by 
Nike’s customer is best supported through a blog model. As a contrast, the Scottish 
Football Club anthology relied upon the wiki model because all users occupied a 
developer/consumer role. Within the wiki model, all users provide the content they 


thought added value to the anthology. 
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In addition to describing the methods used to collect SATCOM system 
performance information, Chapter 2 also identified the process’s stakeholders. Many of 
these stakeholder’s interests will be relevant to NMT and they must be considered when 
developing and fostering an NMT online community. Fortunately, as was the case with 
the Nike and Scottish Football Club endeavors, the members of the NMT community can 


be identified and designated consumer, developer or contributor community roles. 


1. System Operators 


System Operators will be responsible for initiating, monitoring, and terminating 
the communication services enabled by the NMT system. As the system users with the 
most NMT system interaction they will be very active participants in documenting 
system performance. In this community, the System Operator will have their own 
experiences to contribute, as well as, benefit from the contributions of others. The 
System Operators should be developers, contributors, and consumers within the 


community. 


2 System Maintainer 


The System Maintainers are responsible for performing preventative maintenance 
in accordance with prescribed schedules, and, in the advent of a failure corrective 
maintenance. The System Maintainers will have a breath of experience from maintaining 
and repairing the NMT system, but will also benefit from similar experiences of other 
System Maintainers and System Operators. System Maintainers should be developers, 


contributors and consumers within the community. 


3. RMC Technicians 


RMC technicians are the first line of defense for both distance and onsite 
technical support. In this capacity these technicians will have experiences and knowledge 
to share with the community, but again, also benefit from the experiences of others. 
However, unlike the System Operators and System Maintainers, the RMC technicians are 


not present when the problem is initially experienced, they, much like the customers in 
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Nike’s blog, participate in the community after the topic has been selected. With this in 


mind, the RMC technicians will be contributors and consumers within the community. 


4. ISEA Technicians 


The ISEA and its technicians are both the last line of defense for system problem 
resolution and the proxy for ensuring the Acquisition Program Office’s USC Title 10 
sustainment responsibilities are fulfilled. In their first capacity, ISEA technicians are 
very similar to RMC technicians in that they have both valuable contributions and benefit 
from the contributions of other community members, but don’t determine the discussion 
theme. However, as the Acquisition Program Office’s sustainment proxy, the ISEA 
technicians will posses information pertaining to the fielding of Engineering Change 
Proposals (ECP). This information will describe the ECP’s purpose, the process for 
performing the ECP, and the schedule for deploying the ECP. For these reasons the 


ISEA technicians will be developers, contributors, and consumers within the community. 


5. Acquisition Program Office 


Per USC Title 10, the Acquisition Program Office is responsible for the NMT 
system’s total lifecycle support. As a result of this statute, the Acquisition Program 
Office manages the program budget and must make decisions on resource expenditures. 
The information collected by the data system will assist in making decisions on the 
expenditure of financial resources. However, the Acquisition Program Office isn’t a 
terminal user nor do they directly participate in the resolution of system problems. 


Therefore the Acquisition Program Office is only a consumer within the community. 


6. System Developer 


With the current AN/USC-38(V) and AN/WSC-6(V) MILSATCOM systems, the 
System Developer, under tasking from the Acquisition Program Office, 1s responsible for 
performing system upgrades and enhancements; it is anticipated the System Developer 
will play the same role in the NMT program. Predominantly these upgrade and 


enhancement efforts are initiated to address system deficiencies experienced in an 
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operational environment. Frequently, the System Developer will not have experienced 
these deficiencies in their lab and are dependent upon field experience to recreate and 
diagnose the deficiency, making the System Developer a consumer. As previously 
discussed, the ISEA Technicians distribute information about ECPs eliminating the need 
for the System Developer to contribute this data to the community. Further, while the 
System Developer is an expert on the NMT system, they have limited knowledge about 
the systems that interface with the NMT, which could be the source of the user’s 
problem. This undermines their utility in directly interfacing with the System Operator 
and System Maintainers for problem resolution. For this reason the ISEA Technicians, 
again, interface with the System Developer when detailed technical system information 1s 
required. For these reasons, the System Developer is only a consumer within the 


community. 
7. Technical Documentation Developer 


The developers of technical documentation are responsible for the initial 
development and updates of the system’s technical documentation. Feedback on system 
performance and the current products is valuable information when developing an update, 
creating a need for the developers to be consumers. But, like ECPs, the ISEA technicians 
introduce documentation changes so there is little information the Technical 
Documentation Developers can contribute to the community; for this reason they should 


only consumers. 
8. Training Curriculum Developers 


The Training Curriculum Developers role is very similar to that of the Technical 
Documentation developers in that they develop and update published materials, in this 
case the training curriculum. Feedback on the operational performance of the system is 
beneficial when performing updates to the training. This feedback allows the developers 
to add material and augment the focus on the training based upon the operational needs of 
the users. Once these updates are completed, they are administered via formal instruction 


in the schoolhouses. There are instances that may necessitate the development of ad hoc 
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On the Job Training (OJT), but like ECPS and technical documentation updates, the 
ISEA Technicians administer this training. As a result, the Training Curriculum 
developers don’t contribute any information to the community and participate as 


CONnSuUMeLS. 


Table 14 summarizes the role the different stakeholders take within the NMT 


community 


Table 14: Data Usage Based upon Community Role 


Symone |X| 
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B. PROPOSED DATA COLLECTION SYSTEM USING WEB 2.0 MODELS 


The mashup model facilitates a tailored selection of applications for supporting a 
community making it a very powerful application in developing and fostering online 
communities. For this reason, the mashup is the model upon which the proposed NMT 
performance data collection system is based. The proposed NMT data collection mashup 
will integrate blogs, aggregation, and wiki pages with supporting folksonomies to enable 
interactions between the NMT stakeholders creating an NMT community. Through these 
community member interactions the performance of the NMT system will collected and 


available for analysis to improve the availability of the NMT system. 
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1. Blog in the NMT Performance Data Collection System 


Blogs will be the central component of the proposed NMT performance data 
collection system. Every NMT community member with a developer role, 1.e., System 
Operators, System Maintainers, and ISEA Technicians, will own a blog. The blog will 
serve as their NMT user log, documenting their experiences and challenges with the 


NMT system. 


In their user log blogs, System Operators will document instances of when the 
NMT failed to perform or anomalous behavior. As previously discussed, these instances 
can cover a broad range, from a simple power cycle to a catastrophic failure warranting 
the need to engage a System Maintainer. Regardless of severity the System Operators 
will be encouraged to blog about anything pertaining to the NMT system, but, at 


minimum will blog about every challenge they face. 


System Maintainers will use their blogs to document every NMT maintenance 
action they perform, either preventative or corrective. For corrective action, the System 
Maintainers will describe the experienced problem, diagnosis process and corrective 
action performed. In the advent the corrective maintenance warrants completion of a 
4790/2K, 1.e., the maintenance action is differed, one will be completed; the 4790/2K 
process will not be replaced by the blog. However, weak on improving the long-term 
availability of the NMT system, the 4790/2K and CASREP are part of a well-established 
system for reporting current material status up the command chain; this is a critical 
attribute of military planning, which cannot be circumvented. Also, it would be ideal if 
the blog posting auto populated the Maintenance Data System (MDS) when a 4790/2K 
was watranted, but there are sites without a software supported MDS, and those with 
software don’t support external file loads. Since the System Maintainer will be blogging 
about the failure prior to making the decision to complete a 4790/2K, a template can be 
created in an attempt to reduce the redundant work. While the 4790/2K cannot be 
eliminated, the blog will eliminate the 4790/2L. In the blog the System Maintainer will 
record both the steps taken to diagnose the failure and the corrective actions taken; these 


blog entries will record the detailed information typically recorded in a 4790/2L. 
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As discussed during the analysis of community roles, the ISEA Technicians liaise 
with the community on system updates such as ECPs, technical document updates, and 
ad hoc training. In the proposed NMT data collection system the ISEA Technicians will 
use their blogs to communicate the details of these updates. For instance, for an ECP the 
ISEA Technicians will make blog postings describing the purpose of the ECP, the 
process for ECP implementation as well as its associated deployment schedule. In 
addition to ECP notifications, the ISEA technicians will use the blog to inform the 
community of highly pervasive defects for which a solution does not yet exist. Also, the 
ISEA Technicians can use their blog to administer ad hoc training on emergent system 


issues or areas needed additional attention. 


The members of the community with a contributor role participate by making 
comments to the blog postings. These comments, and in turn the comments made in 
response to other comments, result in an ongoing dialogue. In the NMT community the 
topic of this dialogue will be operation and repair of the NMT system. For instance, if a 
System Operator makes a posting to their blog describing a particular problem other 
community members, at all levels, can provide feedback or suggestions via comments in 
an attempt to resolve the problem. This creates the opportunity for the entire community 
to assist in resolving a single site’s issues. If the System Operator, with the assistance of 
the community member’s comments, is unable to resolve the issue, they will, as in the 
current problem resolution process, notify the System Maintainer. Once notified, the 
System Maintainer will make postings to their blog describing their diagnosis and 
corrective action efforts. The other members of the community can assist in resolving the 
problem by making comments to the System Maintainer’s blog. If further assistance, is 
required from the RMC Technician, and subsequently the ISEA Technicians, these 
contributions will be made via comments to the System Maintainer’s blog; this will 
ensure the entire failure thread from discover through diagnosis to corrective action 1s 


documented in one location. 


Despite their participation via comments to System Operator and System 
Maintainer blogs, the RMC technicians and ISEA technicians must be notified if the 


problem proves too challenging for the System Maintainer to resolve for a couple of 
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reasons. First, the real opportunity for the blog and comment architecture is that it 
provides the potential for every member of the community to provide their knowledge in 
assisting the System Maintainer, but it is not anticipated that every member of the 
community will contribute or even review every posting at the relevant time; this 
includes the RMC and ISEA technicians. Explicit RMC and ISEA notification of 
irresolvable problems ensures they are engaged and participative in the process. Second, 
there are instances where onsite support by these technicians will be required, and in 
order to justify the commitment of resources the RMC and ISEAs need to be positively 
engaged in the process. However, again, regardless of how the RMC and ISEA 
technicians contribute, whether through general community participation, requested 
distance support or onsite support, all contributions will be made as a comment to the 
System Maintainers blog. For instances where rich interactions take place, such as 
telephone calls or onsite technical assists, the blog comment will still be used to 
document the diagnosis and corrective action, serving as the technicians trip report. 
Again, this consolidated blog and comment approach ensures the entire event from initial 
experience to resolution is documented allowing community members to analyze the 


entire failure event. 


Since all NMT community members are consumers, the blog postings and 
comment threads are available to everyone within the community for their desired 
purpose. For the System Operators, System Maintainers, RMC Technicians, and ISEA 
Technicians this repository of blog postings and comments provides a knowledge 
repository of problems and associated solutions to be utilized as a tool for resolving their 
own problems or facilitate ad hoc training. In a reactive fashion, community members 
can explore this repository seeking out other member blogs or comments that might add 
insight to the root cause and corrective action for the problem they are currently 
experiencing. In a more proactive fashion, community members are able to survey other 
member’s blog postings and comments to provide their contributions or simply increase 
their own awareness of the NMT system’s behavior and problems. Members with only 
the role of consumer, i.e., Acquisition Program Office, System Developer, Technical 


Manual Developer, and Training Curriculum Developer, will use the blog postings and 
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comments to understand the true performance of the NMT system. The blogs and 
associated comments will provide a rich trove of data for these consumer only roles to 


survey and identify problems with system, technical documentation, or training. 
2; Aggregation in the NMT Data Collection System 


The blog and comment architecture provides an opportunity to leverage the 
knowledge of the entire community towards the resolution of a single member’s problem. 
However, given the temporal nature of problem occurrence and resolution this 
opportunity is highly perishable if the community doesn’t provide support in a timely 
manner. This challenge is resolved through subscription and aggregation. Members will 
subscribe to each other’s blogs. Via these subscriptions, members will receive 
notifications of another member’s blog postings informing the community of their 
problems. These notifications are collected and presented for scanning by the subscriber 
via their aggregator. For postings, the recipient feels they contribute too or are interested 
in, the aggregator will provide a Web link allowing quick access to the full blog posting 


for reading and comment. 


The number of blogs that one member subscribes to is an individual preference, 
but, at a minimum, a member will subscribe to the blogs within their Carrier Strike Group 
or Expeditionary Strike Group and those of the ISEA Technicians. Every member 
subscribing to every other member’s blog would provide the greatest opportunity to share 
and leverage the knowledge of the community. However, with a community in excess of 
1,000 members, it is likely the mass subscription model would overwhelm members; the 
intention of the data collection system is not to have members of the NMT community 


blogging all day. 


Subscribing to the blogs within their Carrier Strike Group or Expeditionary Strike 
Group allows community members to actively contribute to the readiness of the NMT 


systems within their operational unit. A subscription to the ISEA Technician’s blogs 


© The Carrier Strike Group or Expeditionary Strike Group consists of an aircraft carrier or amphibious 
air platform and its escort ships. This is the configuration that supports Navy deployment cycle and 
operational organization. 
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allows all community members to stay current on the latest system developments. It is 
envisioned RMC Technicians will subscribe to the blogs within their Area of 
Responsibility (AOR). ISEA Technicians, as the de facto system experts, willsubscribe 
to all community blogs. Regardless of the subscription model employed, all members 


will be encouraged to subscribe to and participate in as many blogs as possible. 


3. Folksonomies in the NMT Data Collection System 


Where the subscription model allows community members to support each other 
in a timely fashion, folksonomies allow the community to organize and search their 
repository of NMT data. Again, folksonomies allow content authors and users to place 
descriptive tags within the content, allowing other users to search and group blog 
postings. For instance, one System Maintainer seeking the resolution to a specific system 
fault by searching on a fault ID would find every entry tagged with the fault ID. Also, 
another proactive community member could be trying to improve their troubleshooting 
skills, by searching postings tagged with an array or terms such as ‘good troubleshooting, 
‘informative’, or even ‘good job.’ With folksonomies, it is possible both the System 
Maintainer seeking an explicit fault and the proactive community member to discover 
and utilize the same posting and comment thread for different purposes. Folksonomy will 
also allow the Acquisition Program Office to truly understand the performance of the 


NMT system. 


Folksonomies provides a truly powerful tool to understand trends within a 
community. As a first order use, the Acquisition Program Office can search the 
community data for pervasive system problems. For instance, the Acquisition Program 
Office could, like the System Maintainer above, type in an explicit problem and find 
every instance of a blog being tagged with this problem. The results of these queries can 
be used to determine how many operational sites and NMT systems are afflicted by a 
particular problem. Also, as a second order use, the Acquisition Program Office can study 
the most popular terms or tags within the folksonomy to gain key insights into 


community. For instance, finding ‘junk’ as one of the most frequently used tags within 
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the community would clearly be indicative of a broad perception problem. Folksonomies 
allow the Acquisition Program Office to identify and understand both the micro and 


macro problems within the community. 
4. Wiki Pages in the NMT Data Collection System 


Wiki pages will be used by the NMT community for access to, correcting of, 
publishing, and creation of technical documentation. As part of their ongoing use and 
NMT interactions, System Operators and System Maintainers will utilize the system’s 
technical documentation. Historically, these documents were provided in a hardcopy 
format, but recently the trend is toward electronic formats. Electronic formats minimize 
storage requirements, reduce the risk and impact of being lost, and support rapidly 
publishing updates. The wiki pages employed within the NMT community support all 
these attributes with additional benefits. With wiki pages all NMT community members 
the can correct errors or generally improve the NMT technical documentation. System 
Operators and System Maintainers as they use the technical documentation in 
conjunction with their NMT system will be able to re-articulate content in a fashion more 
conducive to the community or add additional information. In addition, the community 
members will be able to augment the technical documentation with new publications 
borne out of ad hoc processes. Due to the rapid publishing characteristics of wiki pages, 


these updates will be made available to all community members almost immediately. 


While the general trend of wiki based products is towards improved quality, there 
is arisk or incorrect information being propagated to detriment of user safety or collateral 
damage to the NMT equipment. This risk will be managed via two controls within the 
community. First, the wiki pages will accommodate two instantiations of the technical 
documentation; one will be the official current revision and the second will be open for 
editing and updating by the community. Having these two versions allows community 
members to reap the benefits of documents edited with the colloquiums and practices of 
the community with a mechanism for comparison to the fully vetted, un-permutated 
version. By surveying the community updates, the Technical Documentation Developers 


will incorporate changes into the official version facilitating a trajectory of constant 
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improvement. While this construct will support more frequent revision changes to the 
official version than the every couple years frequency supported by the AN/USC-38(V) 
and AN/WSC-6(V) programs, the updates will not be as frequent as the near immediate 
input the community can provide. As a second precaution against detrimental 
dissemination of incorrect information, an ISEA Technician will be assigned as an active 
reader, and if need be, editor of the community updates to technical documentation. Just 
like blogs, wiki pages can be subscribed to, which provides a notification mechanism for 
the ISEA technician to follow updates to the community wiki. Upon receipt of an update 
notification, the ISEA technician will review the updated content looking for unsafe 
practices or technically incorrect information, especially information that describes 
practices which will cause damage to the NMT system. If such content is discovered, the 
ISEA technician will make appropriate edits and notify the publisher of the information 


the motivation for making the changes. 


Lastly, like the blogs, the wiki will be supported by the community folksonomy. 
This, again, allows the community to tag and search the technical publications in terms 
and context that is relevant to their need. A community member will be able to query the 
blog postings and wiki pages providing a rich repository of data to assist in correcting the 


system deficiency. 


C. EVALUATION OF THE PROPOSED DATA COLLECTION SYSTEM 


The proposed NMT data collection process utilizes Web 2.0 technologies, 
specifically a mashup model. The method for evaluating the merits of this system’s 
ability to improve the NMT system availability is against the ideal system. Again, this 
ideal is constructed by examining the parameters affecting NMT availability; the ideal 
system is a system whose attributes address all these parameters. Further, since the 
current system planned for inheritance by NMT was evaluated against the ideal in 
Chapter III of this thesis, the ideal is an intermediary for comparisons between the two 
systems. The sections below evaluate the proposed system against the ideal system’s 


nine attributes. 
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Table 15: Ideal Attribute 1 


Item # Data Recorded Data Customer Purpose Ao Parameter 
Affected 


Acquisition Program 
All system anomalies Office and System System Improvement 


Developer 





The members of the NMT community exposed to system anomalies occupy a 
developer role within the community. As developers, these members, 1.e., System 
Operators and System Maintainers, own blogs to which they post their experiences with 
the NMT system. Other members of the community participate by providing comments 
to the owner’s posting or other’s comments. There are no constraints, e.g., field sizes, to 
the amount of data placed within a blog posting allowing for a detailed description of the 
failure event. In addition, the dialogues enabled through comments may divulge 
information inadvertently omitted. As a result, these blog and comment threads record 
rich detail about system anomalies for consumption by community members including 
the Acquisition Program Office and the System Developer. Further, the folksonomy tags 
used to describe and organize the data facilitates interrogating the repository in a 


multitude of fashions 


The proposed system under performs the ideal in that data collection is dependant 
upon the System Operators and System Maintainers to make blog postings. However, this 
risk 1s managed by two factors. First, the System Operators and System Maintainers will 
find that assistance from the community makes their job of ensuring their NMT supports 
its mission easier. Member comments allow a troubled System Operator and System 
Maintainer to restore their NMT to operational status in a more efficient fashion, but they 
can’t receive this assistance until the community is notified of their problem through a 
blog posting. This incentive competes against a tendency to forgo making a blog posting. 
A second, less positive factor, is that keeping a user log will be a requirement of their job 
as a System Operator and System Maintainer. A mandate like this certainly does not 


guarantee success, but it will help. 
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Table 16: Ideal Attribute 2 


Item Data Recorded Data Customer Purpose Ao Parameter 
Affected 


Acquisition Program Training Curriculum and 
All system anomalies Office and System Technical Publication 


Developer Updates 





Surveying the rich repository of blogs and comments provides insights toward 
improving the training curriculum and technical publications. With an understanding of 
the real problems affecting the operation of the NMT terminal and the common mistakes 
make by the System Operators and System Maintainers the Training Curriculum and 
Technical Publication Developers can augment and improve their products. For technical 
publications, the community wiki pages allow the community to improve these products. 
Within the technical publications the community members will be able to correct errors, 
add additional information or simply put a process in more familiar terms. There is no 
constraint to the number of times these edits can be made and once made they are 
immediately available to the rest of the community. These community updates will 
further assist the Technical Publication Developers in their updates of the official 
revision. These updates will be based on both the analysis of the blogs, comments and 
user updates to the community publications. This facilitates revisions to the official 


publications at rates much faster than currently supported. 


Table 17: Ideal Attribute 3 


Item Data Recorded Data Customer Purpose Ao Parameter 
+ Affected 


Acquisition 


Program Office OBRP requirement 


All system anomalies 
and System updates 


Developer 
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When a site does not have the needed spare part in their OBRP complement, a 
large availability penalty is incurred as the site waits for a replacement part. The 
proposed data collection system retains the 4790/2K process, which documents 
maintenance deferrals, because of its necessity in reporting current materiel status to the 
chain of command. Information from the blog postings will also allow the Acquisition 
Program Office and System Developer identify candidate components for inclusion into 


the site OBRP compliment. 


Table 18: Ideal Attributes 4 through 6 


Data Recorded Data Customer Purpose Ao Parameter 


Affected 


System User 
All system anomalies Peer-to-Peer Training 
Community 


System Maintainer 
System Maintainer Failure Community and 
Peer- to-Peer Training 
Diagnosis and Corrective Action RMC and ISEA 


Technicians 


System Maintainer 
Community and Guru-to-Expert 


RMC and ISEA Training 


RMC and ISEA Technician 


Failure Diagnosis and Corrective 


Action 2 
Technicians 





Community members with developer roles record their NMT experiences through 
postings to their blogs, and community members with contributor roles interact with 
developers and other community member by commenting on the blog postings. These 
postings and interactions facilitate training amongst all levels of the community. The 
initial opportunity for training is through the dialogue enabled by the blog and comment 
process. The blog Web 2.0 model allows community members to post their 
understanding of a problem and receive feedback from other community members, and, 
in an iterative process, provide comment back. This iterative process can continue for an 
indefinite period of time, allowing a rich discussion and ad hoc training session. The 


second opportunity is a by-product of archiving all these discussions. A community 
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member needs neither to participate nor be present during the original conversation to 
learn from it. The archived posting and comment dialogues serve as lesson transcripts 


available for all member of the community to discover and utilize. 


Table 19: Ideal Attributes 7 through 9 


Data Recorded Data Customer Purpose Ao Parameter 


Affected 


Acquisition 
Failure Diagnosis and Corrective Program Office 
System Improvement 
Action and System 
Developer 


Acquisition _ 
Training Curriculum 


Failure Diagnosis and Corrective Program Office 


and Technical 
Action and System 
Publication Updates 
Developer 


Acquisition 
RMC and ISEA Technician Training Curriculum 
Program Office 
Failure Diagnosis and Corrective and Technical 
and System 
Action Publication Updates 
Developer 





The proposed NMT performance collection system facilitates the Acquisition 
Program Office and System Developer’s efforts to improve MTTR by collecting 
information from all community members and allowing the same members to make 
revisions to the technical documentation. In their blogs, System Maintainers will discuss 
problems experienced while diagnosing failures and performing corrective action. The 
Acquisition Program Office and System Developer can survey the blogs, looking for 
those repair problems by frequency and/or level of intensity of frustration. In addition to 
the blogs, the community updated wiki pages will provide insights into areas where the 
system can be improved, as well as improve the official technical documentation on a 
much more frequent basis than currently supported. One addition users will make to the 


wiki pages are ad hoc work-around procedures created to circumvent either NMT system 
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deficiencies or inadequacies in the technical documentation; this is critical information 


towards improving the MTTR performance of the NMT system. 


The obvious purpose of improved training is to decrease the instances of operator 
error and lessen the impact of these instances. With System Operators and System 
Maintainers documenting their NMT experiences, an opportunity 1s made for the 
Training Curriculum Developers to survey their experiences and find frequent pitfalls. 
This broad canvas of the user community will allow updates of the training materials. In 


addition, these blog transcripts could provide case studies for use in a classroom setting. 


For those instances of operator error, the blog and comment model limits provide 
assistance to limit the impact. The entire NMT community can assist an NMT user in 
extremis via a comment once they post to their blog. This broad support will decrease 
the time the troubled user languishes trying to identify the cause of the problem. In 
addition, discussing and explaining the process tends to change perspectives and facilitate 


insights. 


Table 20 scores the proposed data collection system against the ideal system using 
the same 1-5 evaluation scheme used in Chapter III for evaluating the current system. 
The proposed system scores very well against the ideal—a 4 in every attribute. Through 
the employment of a Web 2.0 mashup model, the proposed data collection system 
addresses all the attributes of the ideal system. However, the proposed system is reliant 
upon the contribution and participation of the users, creating a risk that a threshold of 
reporting will emerge. It is believed this threshold—e.g., an arbitrary severity of a 
problem that triggers reporting—will decrease as members see increasing value in the 
community, and the proposed system encourages and does not preclude full problem 


disclosure, but there is no guarantee all problems will be captured. 


Also depicted are the scores for the current data collection, allowing for a 
comparison between the current and proposed system. The proposed system is equal to 
the current system in one attribute and superior in all others. The fundamental reason for 
this performance is the ethos of Web 2.0, participation. The current system places 


limitations on both who can contribute and with whom the information is shared. For 
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instance, in the current system, the users with the most interface time with the NMT, the 
System Operators, do not have a feedback mechanism on the performance of the system. 
In addition, the most detailed information about the corrective action, the 4790/2L, is 
seen by no one outside the operational site where it was produced and, potentially, by no 
one other than the System Maintainer responsible for its development. By contrast, the 


proposed system removes these data contribution and consumption constraints. 


Also, in the spirit of participation, the proposed system respects and leverages the 
intelligence and knowledge of the System Operators and System Maintainers. The 
knowledge and expertise of these trained U.S. Navy communicators is dismissed by the 
hub-and-spoke architecture of the current model. The data recording and problem 
reporting process is purely hierarchical. A problem at one level proceeds to the next and 
then the next until it arrives at the final organization responsible for ensuring the system 
works; never are the peers encouraged or permitted to contribute. The proposed system 
still provides the hierarchical mechanisms, but also facilitates a “barn raising” ethos 
amongst the community. Community problem solving allows the Navy to leverage and 
utilize the investment made in these professional communicators, and increase the 


availability and readiness of NMT’s critical communication capabilities. 


Table 20: Summary Score of Current and Proposed vs. Ideal System 


Item | Data Recorded Data Purpose Ao Current | Proposed 
# Customer Parameter | Score Score 


Affected 


Acquisition 


Program 
All system System 
Office and 
anomalies Improvement 
System 


Developer 





qT 


Data 


Customer 


Data Recorded 
# 





Purpose Ao Current | Proposed 
Parameter | Score 
Affected 


All system 


anomalies 


All system 


anomalies 


All system 


anomalies 


System 
Maintainer 
Failure Diagnosis 
and Corrective 


Action 


RMC and ISEA 
Technician 
Failure Diagnosis 
and Corrective 


Action 


Failure Diagnosis 


and Corrective 


Action 


Acquisition 
Program 
Office and 
System 


Developer 


Acquisition 
Program 
Office and 
System 


Developer 


System 
User 


Community 


System 
Maintainer 
Community 
and RMC 
and ISEA 


Technicians 


System 
Maintainer 
Community 
and RMC 
and ISEA 


Technicians 


Acquisition 
Program 
Office and 
System 


Training 





Curriculum 
and 
Technical 
Publication 


Updates 


OBRP 
requirement 


updates 


Peer-to-Peer 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 


In employing the Net-centric Warfare strategy, the U.S. Navy and the broader 
Department of Defense estimate technological advances in sensor and networking 
technologies will result in a more flexible, efficient, and potent force than previously 
achievable. The process of realizing this vision has created a critical reliance on the 
systems that deliver these technologies or, in the case of the NMT, enabling technologies 
like bandwidth. With regards to bandwidth, this reliance manifests in Operational 
Availability; the expectation that NMT reliably provides critical off-ship bandwidth is 
very high. However, despite the expectation, from a requirement perspective, the 
availability spread from threshold to objective is quite broad: 90% and 99% respectively. 
To achieve operational performance closer to the objective requirement commensurate 
with expectations, the NMT system will use techniques employed by previous generation 


systems. 


These techniques were developed in an environment of higher redundancy and 
lower criticality. When applied to NMT, they increase risk to both the operation of the 
NMT system and ultimately the Navy’s employment of Net-centric Warfare. These 
current performance collection techniques make understanding and assessing the NMT 
performance a very opaque process. The process creates silos of information with narrow 
or no distribution, resulting in users who, independently and redundantly, solve system 
problems to the detriment of bandwidth availability. These silos also constrain the 
information that the Acquisition Program Office can use when making investments 
toward improving the NMT. Rather than making these investment decisions on a 
thorough understanding of the system, they will be made in response to the loudest 
complaints, using a politically charged feedback loop. This does not mean that these 
legacy techniques fail to support the NMT, but it does increase the risk and—in light of 


the opportunities provided by Web 2.0 technologies—this risk seems unnecessary. 


dS 


The Web 2.0 mantra of individuals creating information for consumption by the 
masses breaks information silos, providing an opportunity to reduce the risk that NMT 
will fail to meet either its requirements or its expectations. For users, eliminating these 
silos provides peer-to-peer training and assistance in diagnosis and resolution of system 
errors, increasing both their knowledge and efficiency. The Acquisition Program Office 
is provided a much broader survey of NMT operational performance, resulting in a better 
investment of resources toward improving the system and its support structures. Web 2.0 
technologies enable an environment for both the user and the Acquisition Program Office 


to contribute, well beyond the legacy methods, toward improving system performance. 
B. RECOMMENDED AREAS FOR ADDITIONAL RESEARCH 


Creating a community of system stakeholders presents a great opportunity for the 
operation and sustainment of the NMT system. As a result, it is highly recommended the 
NMT program pursue these opportunities to exploit the benefits available to the users, the 
Acquisition Program Office, and the U.S. Navy. However, the world of Web 2.0 
technologies is a dynamic, rapidly evolving environment creating new twists on 
deployment models and lessons learned on their deployment. With this in mind, it is also 
recommended these evolutions be followed, and the lessons learned surveyed, to support 


further research prior to or part of the employment of Web 2.0 models. 


1. The Appropriate Size and Scope of Online Communities 


This thesis examined the employment of a mashup model to foster a community 
of NMT users with the objective of improving system performance. This examination 
demonstrated an opportunity for this model to exceed the performance of the current 
system. It 1s reasonable to assume this opportunity is not unique to the NMT system, 
meaning a similar benefit could be enjoyed by other systems. While the appeal of Web 
2.0 technologies to systems other than NMT is clear, it is not clear how these benefits 
should be realized when multiple systems are considered, versus the single system, as 


contemplated by this thesis. 
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The NMT system will not be the only system its users will be responsible for; 
rather, the NMT is one of several systems used by these IT and ET sailors. This raises 
questions about the size and scope of the community when the users have multiple 
responsibilities. Is it better to establish a single community for each system, an 
overarching community encompassing all systems, or a hybrid? On one hand, 
participation in many communities may prove burdensome for the users and, on the other 
hand, a broad community may lose focus, diminishing its affectivity. It 1s recommended 
more research be performed to answer the size and scope questions of online 
communities such that the opportunities provided by Web 2.0 technologies can be 


broadly realized. 
2: Methods to Initiate and Grow Online Communities 


Once established, the mashup model community proposed for the NMT system 
provides clear incentives for user participation; it makes their job easier. Meaning—once 
the community has a broad membership of stakeholders sharing information and ideas— 
the value is clear; with these incentives, the community is self-perpetuating. Web 2.0 
technologies thrive on content, the more they have the better they are. However, prior to 
reaching this critical mass, the value to users may not be clear, undermining the 
incentives and the establishment of momentum. For instance, when the community is 
established, the searchable archive of blog dialogues will be very anemic, as will the 
number of blog subscribers able to rush to the aid of a struggling colleague. As the 
massive size and growth of YouTube, Facebook, and the others demonstrate, these 
communities can be extremely self-sustaining once a critical mass 1s achieved. Failure to 
reach the critical mass will undermine the credibility of the effort and significantly 
diminish or eliminate the value they provide. As a result, careful research and planning 


must be applied to develop strategies for the initiation and growth of these communities. 
3. Security Concerns with Online Communities in a Defense Application 


Information is of critical importance, particularly in the context of military action. 


Victory is typically an unassailable result for the combatant with knowledge superiority. 


7T7/ 


Reflecting this, the key benefit within the Net-Centric Warfare doctrine is information 
dominance. Recognizing this criticality, a system of classifications and compartments 
has been established to protect information. In contrast, the Web 2.0 philosophy requires 
frequent and broad sharing of information to realize its benefits. The major reason the 
proposed NMT mashup model has the potential to exceed the performance of the current 
system 1s that it releases information from silos providing utility to all NMT community 
members. This sharing of information creates the potential for a dramatic improvement 


in the methods for supporting the NMT system, but it may also create a security risk. 


As its foundation, the United States classification and compartment system is 
rooted in the concept of “need to know.” It is possible that, during the highly 
collaborative discourse of an online community, individuals without a need to know will 
be inappropriately exposed to sensitive information. This raises a challenging question 
concerning online communities: How do you ensure participants share enough to realize 
the available benefits, but not share too much so as to compromise critical information? 


Or, perhaps an even more difficult question: Can this point be determined? 


With maxims like “loose lips sink ships,” information security 1n casual contexts 
has always been a challenge, but the ease of storage and dissemination made possible by 
electronic files compounds the challenge. The challenges presented by Web 2.0 
technologies may be similar to those posed by e-mail and Web 1.0 considerations, for 
which there are ongoing efforts to manage the compromise of information. Regardless, 
to understand the problem space, as well as find the answers to the questions posed 


above, further research must be performed in these areas. 


78 


LIST OF REFERENCES 


Clark, Vern. “Projecting decisive joint capabilities,” United States Naval Institute 
Proceedings, vol. 128, 2002, 32. 


COMFLTFORCOM INSTRUCTION 4790.3 Rev B, “Joint Fleet Maintenance Manual,” 
http://www.submepp.navy.mil/jfmm/jfmm2.htm (Accessed May 2009). 


Giles, Jim, (2005, December). Special Report Internet encyclopedias go head to head. 
Nature, 438, 900-901 


Hof, Robert, (2003, December). “Reprogramming Amazon.” BusinessWeek, 


http://www.businessweek.com/magazine/content/03_51/63863115_mz063.htm 
(Accessed July 2009). 


Joint Chiefs of Staff Joint Pub 6-0, “Doctrine for Command, Control, Communications, 
and Computer (C4) System Support to Joint Operations,” Washington DC, May 
1995. 


Mader, S. (2008). Wikipatterns. Indianapolis: Wiley. 


NAVSEA INSTRUCTION 4790.8B, “Ships’ Maintenance and Material Management (3- 
M) Manual,” Washington DC, November 2003. 


NSG INSTRUCTION 4000.1D, “NAVSECGRU Logistics Management Manual,” 
Washington DC, April 1998. 


OPNAV INSTRUCTION 3000.12A, “Operational Availability Handbook, A Practical 
Guide for Military Systems, Sub-Systems and Equipment,’ Washington, DC, 
June 2003. 


Pink, D. (2005, December 11). Folksonomy. The New York Times, 
http://www.nytimes.com/2005/12/1 1/magazine/1 lideas1-21.html?_r=1 (Accessed 
July 2009). 


PMW 170, “Navy Multiband Terminal Navy Training Systems Plan,” San Diego, 
California, November 2008. 


Sankar, S., Bouchard, S., (2009). Enterprise Web 2.0 Fundamentals. Indianapolis: Cisco 
Press. 


79 


THIS PAGE INTENTIONALLY LEFT BLANK 


80 


INITIAL DISTRIBUTION LIST 


Defense Technical Information Center 
Ft. Belvoir, Virginia 


Dudley Knox Library 
Naval Postgraduate School 
Monterey, California 


Dr. David H. Olwell 

Department of System Engineering 
Naval Postgraduate School 
Monterey, California 


81 


